2.0.6.3 in advanced beta testing

MScan and MView 2.0.6.3 32- and 64- bit are in advanced beta testing and should be released soon.

New Features in 2.0.6.3.

Heat display. Functional imaging of  fast calcium transients in live tissue is a very important application of two-photon microscopy.  Monitoring fluorescence changes in real-time provides essential feedback on how the experiment is running. MScan allows plotting the intensity of multiple regions of interest across time even in resonant scanning mode. In release 2.0.6.3, MScan introduces a new way to visualize active areas by displaying a live heat map with “hotter” colors  representing pixel intensities greater than those of a "baseline" frame. The map is superimposed over the current frame. Regions whose activity is below a pre-set threshold will be seen through the heat map. The heat map will facilitate spotting patterns of spatial activity. MView 2.0.6.3 also allows overlaying a heat map above frames, which can lead to the creation of visually appealing movies.

Data file recovery. Programs like MScan or the computer running them can crash unexpectedly, resulting in damaged data files. MView 2.0.6.3 can detect when a file generated by MScan 2.0.6.3 is corrupted and offers the possibility to repair it via a window allowing users to correct or enter missing metadata information.

Minor improvements in MScan 2.0.6.3.

• Increased limit on number of frames that can be averaged during live display or stacks. Up to 200 frames can be averaged on a rolling basis and displayed when taking movies.
• Resonant mode: frame rates of 496 fps (512 x 32 lines) and 565 fps (512 x 28 lines) have been qualified.

Cloud-based storage in MCS 2.0.6: setting up MScan and MView to access the cloud

In MScan and MView, each user (chosen or created from the User window shown when the programs start) has his or her own cloud settings.

Click on the Settings/Cloud Storage Settings menu item on uppermost window of MScan or MView.  

For Amazon S3:

Choose Amazon S3 for the service and  enter the Access key ID and the Secret key ID.

For Microsoft Azure

Choose Microsoft Azure for the service and enter the Storage Account Name and the Primary Access Key.

Cloud-based storage in MCS 2.0.6: introduction to S3 and Azure

Amazon S3 (Simple Storage Service) and Microsoft Azure services are very similar in the way they operate. Their services can be tried for free, either for a limited time or for a certain amount of data transfer. In addition, data files located on the cloud are organized in a similar way (see table below). The the top level is called an account, which would be similar to a disk drive. The account holds either buckets or containers, which are very similar to disk folders or directories. Buckets or containers contain objects or blobs. For all practical intent, objects or blobs are MCS data files. A major difference between a file system on a disk and the S3 or Azure cloud storages is the fact that buckets or containers cannot contain other buckets or containers, only objects or blobs.

Your Amazon S3 Microsoft Azure

computer

Drive Account Storage Account

Folder Bucket     Container

File         Object Blob 

An Azure storage can store up to 200 TB of data. Blobs can be as large as 200 GB.

MScan allows you

• to create new buckets and containers
• and directly upload data files to one of these "folders" at the end of an imaging session.

MView is capable of

• dowloading data files on the cloud to a local folder on your computer
• create new buckets or containers.
• upload files to the cloud
• delete files in the cloud

Cloud-based storage in MCS: setting up your S3 or Azure account

For Amazon S3:

1. You need first to set up an account. Go to:

Amazon S3: http://aws.amazon.com/s3/

2. Once you are activated, you will receive a confirmation e-mail. Follow the instructions in the e-mail to set up Access Identifiers. Use the Web-based AWS Management Console to try creating a bucket, manually upload a file etc… Click on the Security Credentials menu item (under the menu item bearing your user name).

The two most important information to retrieve from your account are:

• Your Access Key ID: it is a 20 character-long string
• Your Secret Key: it is a 40 character-long string

Copy these two values somewhere (on paper, in a text file or in an e-mail) to register later in MScan and MView.

For Microsoft Azure:

1. You need first to set up an account. Go to:

Microsoft Azure: http://www.windowsazure.com/en-us/

This requires an account name and a password.

Log into the Windows Azure Management Portal.

Click on the NEW button at the bottom of the navigation pane 

Click on DATA SERVICES menu item on the left of the window, then click on STORAGE, QUICK CREATE.

Enter a name for the storage account in the URL input box (e.g., jimtwophotonfiles), select a location for the file repository (e.g., West US) and enable/disable geo-replication.

2. Once the storage account is created

Click on the Manage Access Key button at the bottom of the screen 

The two most important information to retrieve from your account are:

• Your Storage account name
• Your Primary Accesst Key

Copy these two values somewhere (on paper, in a text file or in an e-mail) to register later in MScan and MView.

Backing up and archiving your multiphoton data: the MCS 2.0.6 cloud-based solution

Optical imaging instruments such as multiphoton microscopes generate large amount of data. Because MOMs are often used to collect two-photon images many times per second over periods ranging from minutes to hours, typical files can routinely exceed hundreds of megabytes. The trend towards producing tens or hundred of gigabytes of data per day is accelerating with the use of video-rate resonant scanning imaging in behavioral experiments. As a resut, the problem of backing up and archiving terabytes of two-photon datasets is becoming particularly acute. One could use a local file server but this requires funds to buy the hardware, expertise to set up the server and some kind of periodic administration. And of course, when the file server is full, someone will have to take care of the problem.

The computer industry has already a solution, called "cloud storage" to safely store large amount of data without having to invest in a costly infrastructure. In exchange for a small fee, Web behemoths such as Amazon, Google and Microsoft offer disk space on their servers to end-users, provide a Web-based interface to administrate accounts and programming tools to upload or download files. The advantages of cloud storage include:

• virtually unlimited or extremely large storage capacity.
• reliability: the data is usually stored in triplicate and, optionally, at sites that are geographically distinct to prevent losing data in case of a catastrophic hardware failure.
• universal availability through an Internet connection.
• flexible pricing that scales up with the amount of data you store.

MCS 2.0.6 now leverages cloud storage capabilities by allowing:

• in MScan, to systematically upload your newly created data files to a user-specific location on the cloud upon finishing an imaging session. You won't have to back up (or forget to archive...) your data files manually anymore.
• in MView, to download data files from the cloud to your local disk storage to perform analysis. Your data is available anywhere and at all times as long as you have an Internet connection to access your cloud account.

MCS 2.0.6 at present supports two services, Amazon S3 (Simple Storage Service) and Microsoft Azure. If you opt for their fee-based plan, Amazon has no restriction on the amount of data you can store while Microsoft now caps the amount of data per storage to 200 TB. However, you can create as many storage accounts as you want. Maximal file size is 200 GB and 5 TB for Azure and S3, respectively.

There are many other cloud storage companies. Google has a service similar to S3 and Azure called Drive, but files cannot be larger than 10 GB, which could be a problem with data collected in resonant scanning mode. For those who are accustomed to Dropbox, it appears that this company uses Amazon S3 as its underlying infrastructure.

MCS 2.0.5 released soon

MCS 2.0.5 is currently being tested for imminent release.


MScan 2.0.5

New features:
(1) NON-RESONANT SCANNING.  The 3DMap window demagnification can be changed to be less than one (i.e., one can zoom in on views).
(2)  NON-RESONANT SCANNING. Photostimulation targets can now be designated in the 3DMap window with a demagnification < 1.
(3) Time-lapse imaging can now be done with intervals less than 1 s (the interval between frames is now set in milliseconds). You can take frames every 500 ms when your frame duration is less than that (for instance ~ 330 ms at the default frame rate of 3.05 fps).
(4) There is now a zPiezo Automation object (of the Piezo type) that allows outputting voltages to the z-piezo controller during scripts.
Features (3) and (4) allow scripts to move the objective with the piezo in time-lapse mode after taking a frame. The idea is to take a frame and while waiting for the next frame, to move the piezo. The interval between each frame will allow the objective lens to settle when performing large z- changes.
Bug fixes:(6) RESONANT SCANNING. There was a bug in generating the waveform controlling the Pockels cell.(5) NON-RESONANT SCANNING. When imaging a subframe (less than 512 x 512) consecutive images are not flipped anymore (subframes are selected by dragging the mouse over a region inside the display area of the Viewer).


MView 2.0.5

Improvements
- When converting to TIFF files, information is now written as an ASCII string (not as UNICODE).

Bug fixes
- Files taken during photostimaging in modes other than region scans can now be open in MView.

The on-line documentation has been updated.

On avoiding TIFF files

TIFF files.

TIFF (Tagged Image File Format) files are used by several notable two-photon imaging software packages to store frame data. TIFF files appeared in the mid-1980s and the latest update, TIFF 6.0 was issued in 1992. Needless to say, TIFF files are woefully outdated - one of the biggest limitations of TIFF file is their size, which cannot be larger than 4 GB.

With conventional scanners, when imaging 512 x 512 frames at 2 fps on two channels, a TIFF file would be full in about 34 min. As I mentioned earlier, this would happen in a little more than 2 minutes in resonant scanning. Writing files in the BigTIFF format could bypass the 4 GB size limitation, but this new file format is not widespread.

Another disadvantage of TIFF (and BigTIFF) files is their internal organization (or lack of). Frame data can be placed anywhere in the file, thus defeating most streaming strategies. In particular, TIFF readers cannot expect the n-th frame to be located at a predictable location related to the frame index - the entire file has to be read until that frame is found, which is arguably a performance problem with large files.

Lastly, the collection of image tags is not extensible so important two-photon scanning information has to be placed arbitrarily in the image description section of the file.

For those who are accustomed to analyzing data in TIFF files, MView provides MCS to TIFF conversion, either for individual files or for batches of files in the same folder.

Structured Storage files.

MCS uses an advanced file format that completely overcome the shortcomings of TIFF files. Firstly, MCS files are based on the Structured Storage technology available under Windows. Structured storage files are like miniature file systems that contain arbitrary numbers of folders called storages which in turn can hold either other storages or internal files called streams.

An immediate advantage of such files is the ability to store analog data (e.g., electrophysiology) in a separate stream alongside imaging data. Individual streams can be as large as 16 TB, so there is little concern that Structured Storage files will run out of space in the foreseeable future. Writing or reading data to or from  the data file is straightforward, thus efficient, during streaming operations. MView has no difficulty opening and reviewing data files larger than 100 GB. In principle you can think of adding all kinds of parallel streams of data, for instance visual stimuli patterns without disturbing the structure of the file.

Similarly, file properties are stored in an independent stream. MScan writes standard file property / value pairs (e.g., microscope lens resolution, stimulation parameters, etc…) but you can add also your own property names and values to the file (e.g., "Animal Line" = "Wfs1-Tg2-CreERT2 : Ai38").

During analysis by either MView or MCSX, file properties can be queried programmatically for property values. You can also visualize the property name / value list of any file without opening it by right-clicking on the file name in any Explorer window.

Structured Storage is a powerful technology used by many professional Windows programs including Microsoft Office, MSSQL, Dassault Systemes SolidWorks and other CAD programs. By using Structured Storage, MCS provides two-photon experimenters with a whole new level of capabilities during imaging and data analysis.