A low-cost super-resolution imaging device based on a cellphone camera and photonic waveguide chips
The preprint for the cellSTORM II device accompanied with a series of applications ca be found on BIORXIV 😊
The compact device features:
- autofocus - automatic coupling mechanism - on-device superresolution imaging - survives cell incubators for several days - performs autonomous imaging over several days - dSTORM with <100 nm optical resolution - costs <1000€ (for single-wavelength imaging) - optical resolution down to 100nm - multiple wavelength can be used (sequentially) - using photonic waveguide chips TIRF is possible
Using SRRF from the Henrique lab it’s possible to quickly increase the resolution even without complicated STORM protocols:
The image of actin labelled HUVEC cells is acquired using a 60x, 0.85NA objective lens. While moving the coupling lens, the varying intensity pattern caused by a changing mode field pattern can be used to increase the lateral resolution of fluorescently labeled samples at low excitation power. This is suitable for live-cell imaging.
With high enough coupling efficiency and laser intensity, the setup enables super-resolution with a final resolution <100nm on a large field of view (FOV). This is well suited not only for educational purposes, but also for research outside the ordinary research and optics labs.
If you want to replicate the device, you can find a detailed description with all necessary parts to order in this repsitory:
We now also have a pictures tutorial with a step-by-step guide on how to build the cellSTORM microscope here
To move the lenses or control the Laser intensity, we relied on Espressife EPS32s. The drawings for the electronic connections as well as the code to control them wirelessly using MQTT can be found here:
Along with the 3D printed parts in the Github-repository, you need a set of mechanical, electrical and optical parts summarized in the BOM below:
Dead Link?: Ebay is not really a reliable source since resellers may discontinue the product. In case you’re looking for a component which can not be found, file an issue here - alternatively: Copy the name under “Details” and search for it in Ebay. (E.g. “3450 300mW 637nm Dot Laser Module TTL/analog 12VDC” site:laserlands.net) Good luck 🍀
Amazon is more reliable, but personally we try to avoid ordering there (Sorry..). We try to update the list with Amazon links
|Laser 635/637 nm||3450 300mW 637nm Dot Laser Module TTL/analog 12VDC||50 €||Laserlands|
|Objective Lens||BRESSER DIN-Objektiv 60x, NA 0.85, 160/0.17||45 €||Ebay|
|2x Mirror||PF10-03-P01 Ø1” Protected Silver Mirror||50 €||Thorlabs|
|2x XY-Stages||XY Axis Manual Trimming Platform Linear Stage Tuning Sliding Table 40/50/60/90mm, 60x60mm||80 €||Ebay|
|Longpass (640)||Chroma 675/50 well suitable for Alexa Fluor 647||200-350 €||Chroma|
|Ocular||MIKROSKOP OKULAR PAAR PERIPLAN H 10 X LEITZ WETZLAR GERMANY||10-90 €||Wie-Tec (Ebay is cheaper, 10x matters!)|
|ESP32||ESP32 ESP32S WLAN Dev Kit Board Development Bluetooth Wifi WROOM32 NodeMCU||7 €||Amazon|
|LED Buk driver||3x SPARKFUN ELECTRONICS INC. COM-13705||18 €||Spparkfun|
|Powersupply||5V, 3A, Various||10 €||Amazon|
|Raspberry Pi + SD + Powersupply + Housing||Raspberry Pi 3 Set /Bundle: 16GB SD-Karte, HDMI, original Netzteil und Gehäuse||70 €||Amazon|
|Optical Pickup||Objektiv Optik Laser KES-400A PLAYSTATION 3 Nicht Funktioniert für Ersatzteil (HINT: Look for KES-400A replacement parts @ Ebay; Sometimes you get 10 pieces for 10€)||1 €||ebay|
|PLA filament||Prusament PLA Prusa Galaxy Black 1kg||25,00 €||Prusa|
|Cellphone+Camera||Huawei P20 Pro 128GB 6GB RAM Single Sim Twilight, TOP Zustand||300 €||Amazon|
|Ball Magents||T::A Kugelmagnete 5 6 10 mm N45 Neodym Magnete NdFeB Menge wählbar extrem stark||10 €||Amazon|
|Screws||M3 DIN912 Hex-Key Screws, HINT: screws need to be ferro-magnetic (i.e. galvanized!), M3 DIN912 at various lenght may work; we usually have tons of them from the @openUC2-Project||10 €||Amazon|
|Micrometer Screw||RS Electronic 0,1 mm resolution||40 €||RS Electronics|
ATTENTION: In case any parts are missing, please file an issue or contact us! We are happy to help out and to improve 😊
As the software we relied on three different APPs for recent Android phones (in our case Huawei P20 Pro).
Briefly summarized and in-detail described below:
|STORM-Controler||Remote control (MQTT) to control things like laser intensity, coupling lens, focus||APP: STORM-Controler|
|STORM-Imager||Control long-term image acquisition by enabling auto-focus and auto-coupling, schedule fluctuating intensity measurements inside incubators||APP: STORM-Imager|
|FreeDCam||Full control over the cellphones camera: RAW and video image acquisition||FreeDCam|
|ImJoy Fiji.JS Learn2Sofi||Fiji.JS Plugin to process temporal stacks made by Wei @ ImJoy|
To control the Lens or Laser using a customized MQTT controler APP, you can visit this repository:
This app allows basic hardware controls:
This is the APP which can record images, control the device and predict a super-resolved result form the camera live stream. The APP can be found here:
It’s just an example how the cellphone maintains the focus. This is done by maximizing the focus metric (i.e. standard deviation over z) as a function of the focus motor position.
This is an example of the SOFI-based superresolution imaging using the neural network mentioned below. We used fixed e.coli labelled with ATTO 647. The fluctuation of the illumination is the result of the discrete mode pattern in the singlemode waveguide chip. The input field changes the intensity pattern.
For the dSTORM experiments we used the open source APP FreeDCam originally developed and mainted by killerink. We provide a modified version for the Huawei P9 and P20 which is used in this work. It enables
To stream the data to a server this manual gives some more detailed information. It requires Android Studio which compiles the c program into an executable. Inside the APP you need to select the IP of the server.
Many thanks to @Killerink to make this work!
We used the following settings:
The python and tensorflow-based neural network to use “SOFI”-like image reconstruction based on
convLSTM2D layers on a cellphone can be found here:
It aims to transfer a stack of noisy low resolution images with varying illumination pattern:
into high resolution images:
It is part of the APP “STORM-Imager” and can be used as an interactive website (Live-Demo). To train the model have a look in the recent
Tensorflow==2.3.0 implementation using
TFLite which can be found in the Github-Reopository here.
Note: It’s in an experimental stage. It’s not optimized for performance. We do not guarantee for proper functionality. Use it on your own risk!
The model was tested on a Macbook Pro using a Chrome browser which runs at ~1fps.
Go to the ImJoy-Plugin Webpage
It will automatically load a sample TIF-Stack which you can find here
Starting the Plugin: Opening the plugin is as easy as selecting it from the ImJoy Icon on the left hand side:
Executing the Plugin: Once you hit the Learn2SOFI button it will propagate the image stack through the network which looks like this after processing:
You can start playing with your own data by uploading you own time stack. It has to have the following dimensions:
Nx=128, Ny=128, t=30
We provide many datasets in a publicly accessible repository. Please have a look at ZENODO [Comming Soon - need to wait for all data first]
Note: Not all the experiments are fully documented. If you need additional experimental parameters, please don’t hesitate to get back to us so that we can add this!
If you have a question or found an error, please file an issue! We are happy to improve the device!
Please have a look into the dedicated License file.
We do not give any guarantee for the proposed setup. Please use it at your own risk. Keep in mind that Laser source can be very harmful to your eye and your environemnt!