Landing page header

Goodbye Flot, Hello Plotly | Other Website Updates

Our most recent website update probably got lost in the excitement about the news that we had started shipping the new MultispeQ v1.0. We have continued to work on the tutorials and data analysis and are happy to announce another update to https://photosynq.org. There are several libraries out there to graph data in the browser and when we started the platform, we decided to use Flot, a great, performance library, which is unfortunately no longer actively developed. Implementing new methods of plotting would have required us to build certain features from scratch. Instead we are now using Plot.ly which opened up its libraries to the public. It not only provides the library, but also its own platform to create and manipulate plots, as well as libraries for popular languages like Python or R. We would like to give you a quick update on what has changed and improved.

New and Improved Tutorials

While some people really like using YouTube videos to learn new technologies, others don’t find them very helpful. So, we updated the tutorials (available in the learn more menu from the top of the page) to include text pages that include step by step directions, pictures and screen caps to get you started with PhotosynQ. Also, you can now download a Getting Started manual in pdf form if you need to refer to the tutorials while working offline! We kept the the video’s for those you like them, you can find them in their own videos tab on the tutorials page.

New Plot Options

Scatter plots are a great way of looking at data down to individual measurements. At the same time it can be hard to see trends in a big cloud of points. We hope these new plotting options will help analyzing your data.

Contour Plots

Contour plot with additional histograms.
Contour plot with additional histograms.

The 2D contour plot allows you to do exactly that. With additional histograms they are a great way to visualize populations within a big data set.

Histograms

2D histogram using the Jet color gradient
2D histogram using the Jet color gradient

In addition to the already existing histograms, you can now plot two parameters as a 2D histogram.

Box Plots

Box plots with all data points.
Box plots with all data points.

Box plots are available now. Like the bar charts, you can use a category as well.

New Parameter – Time of Day

We added the parameter Time of Day, showing the time, when the measurement was recorded as a number (e.g. 1:30 pm = 13.5). This will be helpful to plot time dependent trends, which could not be visualized using a regular timescale, which could be the case when measurements were recorded over multiple days.

New Data Selection and Plot Capturing

With the new ways to plot data comes the ability to select a set of data points in a scatter plot using the box or lasso selection. It can be used to generate a selection based Series, or the data can be saved with the measurement identifiers as well as the meta data including time and the project questions and answers. The little camera icon enables you to save the current graph as an image (png or jpeg), including the graphs’ legend.

Selecting data points using the lasso tool.
Selecting data points using the lasso tool.

Using Python and R

Analyzing big datasets using the data viewer inside the browser might be difficult. Or you might want to do your own statistical analysis or plot the data in a way that is not available. For this reason we made packages for Python and R to help you get the most out of your data.

Go to https://photosynq.org/software to find the links to the packages and how to use them.
The tutorial section (https://photosynq.org/tutorials) has some examples for Python and R on how to get your data analysis started.

And There is more to Discover…

  • On your user page, you can now see your Network. Your Network includes people who collaborate with you and have joined one or more of your Projects, as well as all Project Leads of projects you have joined.
  • Tags and Project Categories are now displayed on your user page.
  • Tags are now links and can be searched. Just click on a tab or type your tag into the search field (e.g. #beans).

What’s No Longer Available

  1. Plots – We said goodbye to a few features, because there was no good way of adding it to the new plotting library or the available options didn’t turn out to be all that useful.
    • The options of plotting spline lines and area charts.
    • “Time (normalized)” is no longer available. It got replaced by the parameter “Time of Day”
  2. Dashboard – To make the dashboard a little easier we decided to remove the following options.
    • The Panel with a pie chart of total submitted and flagged. The totals are added to the other data quality panels.

We hope that these new features will help you to analyze your collected data.

~ Sebastian

Landing page header

PhotosynQ | Website Update

As we get ready to release the MultispeQ v1.0 we have been working hard in the background to provide an improved user experience. This includes visible changes, as well as a lot of changes in the background. Over the last couple of months we have updated the website, adding and updating features. Here is a brief summary of the biggest changes to https://photosynq.org.

1. Search

Now you can search not only through Projects, but also through Protocols, Macros, Discussions and Users. The searchable content is scored, so we can hopefully provide more relevant search results at the top of the list for you.
Further you can filter your search results for Projects, Protocols and Macros by their associated category.
Go to https://photosynq.org/search to try it out.

2. Discover

If you are new to PhotosynQ or you don’t know what to search for, take a look at the improved Discover page. Now you can browse Projects, Protocols, Macros and Users, and filter the results by category.
Go to https://photosynq.org/discover and get inspired by the work of others.

3. Help Getting Started

We have added a lot of content that we hope will help users get started with PhotosynQ and collect quality data. This content includes new and updated tutorials, a help center, and frequently asked questions.

A. Tutorials

Over the summer we started to make some new tutorial videos, helping you to get started with the platform and the new MultispeQ v1.0 instrument. We added some advanced tutorials on how to create a measurement Protocol, a Macro and how to do a basic analysis outside the platform, using R. Also, check out the ‘further information’ sidebar for downloadable pdf tutorials.
Go to https://photosynq.org/tutorials to see what we recorded for you.

B. Help Center

Along with providing more detailed tutorials, we have added a new help center that will guide you through: managing your account settings, creating a project, viewing your data, creating and editing protocols and macro’s and much more. We have also moved the blog over to the ‘learn more’ tab of the menu bar and replaced it with a link to the help center.

Changes in the navigation menu

C. Frequently Asked Questions

We often get asked questions directly by email and not through the forum. Because of that, we have started a collection of the most frequent and pressing questions. Over time we will add more questions and answers. Go to https://photosynq.org/faq to check out what we have gathered so far.

4. Data Viewer

We worked on the Data Viewer, improving existing features and adding new ones including adding thresholds and adding custom data in addition to the measurements you have already submitted.

Plotting data in the data viewer

  1. Now you can use create plots with up to four dimensions, adding marker size and marker color. The second y-axis has proven to be less helpful has been removed.
  2. Thresholds allow you to look into a subset of data, after you have filtered your dataset into different series. You can select thresholds for one or multiple parameters.
  3. The parameters are now sorted into Primary Parameters, Project Questions, Custom Parameters (if available) and Advanced Parameters, making it easier to find the parameter you are interested in among the many provided. Furthermore, this order of the parameters applies now to the columns in the spreadsheet view as well.
  4. New range sliders and a calendar are available to allow you to select a specific time and date range more precisely, especially when you have long term projects, or a lot of data over a short period of time.
  5. If you are looking at a single measurement, you can now easily navigate to the next / previous one. The small map indicating the measurement location now shows a bigger section with topography as well as latitude and longitude.
  6. New data types can be used, allowing you to display arrays as sparklines when you are looking at a single measurement. Furthermore, colors can now be defined as hexadecimal, RGB and RGBa, allowing you to use them as coloration in your plots or for your map.

5. Additional Data

Some experiments are going beyond data collection in the field with the MultispeQ or other instruments. Now you can add more information to each measurement after you took them (e.g. yield), as long as you are the project lead. This data is kept separate from the original measurements and can be altered at any time, in case you want to change, remove or add information.

additional data upload

6. User Activity

Previously, your user profile mainly presented a number of stats about your activity on PhotosynQ and the Projects you are involved in, either as a Project Lead or a Collaborator. Now we have an Activity feed, which displays the last 100 important events relevant to you on PhotosynQ. You can see if others contributed to your Project, if somebody posted a comment or question to one of the Projects you are involved in and of course everything you did, so you can more easily navigate to what is important for your workflow.

7. In the Background

One of the biggest changes is the work done in the background. Before the update, the original data for a Project was loaded into the Data Viewer and processed on your computer. Now, we do this step on our server and send you only the results from the calculations. As a result, the amount of data sent to you is reduced to 10–15% of its original size, allowing faster access to Project results.
Updating the calculations for a parameter? No problem, since your data gets re-analyzed when you view it after the changes, so you are always up to date.

First 20 MultispeQ devices delivered from factory!

We are continuing to make progress on shipping the MultispeQ v1.0.  We now have the first 20 devices from the factory.  These are exactly (!) like the ones we’ll ship to those who pre-ordered.  And we now have all parts in stock to complete all orders and then some.

We are now evaluating these initial 20 devices, adjusting our existing calibration routines, and field testing them to make they hit our high standards.  We did as much of the testing as possible prior to receiving these 20 devices on hand-built versions with 3D printed cases, but due to small changes in manufacturing it wasn’t possible to finish all calibration routines and test all tolerances. Here’s two such examples: 1) 3D printed cases are notoriously slopped in comparison to injection molded ones, so mechanical components needs to be retested (like the bushings for the clamp mechanism); 2) the case material (ABS) has a different light absorptivity than the 3D printed cases we used before, and that changes the absorption and chlorophyll content calibrations.

In addition, we’re fixing some good old fashioned errors, like we ordered a bluetooth module but were shipped the wrong one (currently being reordered), and the USB 3.0 port wasn’t soldered on the board completely, causing confusion during initial calibration (a new soldering routine is being developed by the factory).

These aren’t excuses, but hopefully give you a sense of the kind of troubleshooting and persistence that’s required to get something that works consistently 🙂

Once we’re comfortable with these 20 devices, we will manufacture the remaining devices and ship.  We have about 2 more weeks until the final parts we had to re-order come in, and it will probably take a few more weeks to complete the calibration and final testing before shipping.  But we won’t ship anything until we’re completely comfortable with the device and it’s efficacy.

We apologize for the long delays, we know it can be both nerve racking and frustrating.  We are getting there, and we think you’ll be happy with the result.  Thanks for your patience and support!

PhotosynQ software features + fixes

Lots of software changes and improvements, and more on the way.  Here’s a list of what’s going on.

Creating a project has changed!

  • First, we completely revamped how you create a new project.  We’ve consolidated the photosynthesis measurements and chlorophyll content measurement into the same protocol to make it easier to find and order.  This new protocol is called “Leaf Photosynthesis” – please use it!  For existing users – NOTE!  Please use the new protocol, but if you absolutely need to find the old protocols (The One v3.0 and Chlorophyll Content III or any others) you can always go to the ‘advanced’ screen for the old interface.
We combined the old protocols together into a single protocol which measures chlorophyll content and all other photosynthesis parameters.
We combined the old protocols together into a single protocol which measures chlorophyll content and all other photosynthesis parameters.
  • You email invite people  to join your project as collaborators or administrators, and have multiple administrators to share the workload of writing results or adding or editing questions.
Send email invites for collaborators, and add administrators so you can share the project-creation workload!
Send email invites for collaborators, and add administrators so you can share the project-creation workload!
  • You can add multiple images to your project and higher quality descriptions.  And there’s tags and categories so other like-minded people can find you and you find them.
Added Categories to more easily find cool projects and protocols
Added Categories to more easily find cool projects and protocols

 

  • We found that directions was underused, so we removed the directions box.  Don’t worry, we’re going to put directions back in later in a much more useful form, but for now if you have directions for collaborators put that in the “Descriptions” box.

 

Android App bugs fixed + new features

The android app (to be released next week) also has some updates. An improved data display screen, better bluetooth connection screens, and a new history feature.  History keeps track of every data point you have collected, so you can identify mislabelled measurements and track your progress through the field.  In the “settings” tab, you can set the app to wait before sending the measurements to the cloud – giving you the chance to delete bad or mislabelled measurements at the end of a day of collecting data.  Once your confident the data is correct and you’ve remeasured if necessary, you can push the data to the cloud.  No more spending months trying to track mistakes by data collectors you can’t contact anymore!

Bigger, cleaner buttons, history function to ensure your data is correct before pushing to the cloud, and save images + notes to any uploaded data point on the fly.
Bigger, cleaner buttons, history function to ensure your data is correct before pushing to the cloud, and save images + notes to any uploaded data point on the fly.

Finally, both apps now automatically stay up-to-date with the website, so no more syncing when you change a project or protocol!  The entire platform should be up to date, all the time – ensuring that as you and other project partners make improvements everyone data, projects, protocols, and discussions are current.

We have a lot of additional news on the hardware front, but I’m waiting for more data to come in before posting about it – so expect that soon.

Updates: Devices available and progress during Beta

Devices Available!

10 devices without CO2 sensors (so no soil respiration measurements) are available on publiclab.org (http://store.publiclab.org/products/multispeq-handheld-fluorometer).  We have quite a few more in the works, so if you miss this batch you can catch the next.

New ways to get a MultispeQ

 

PhotosynQ Share

We’ve learned two important things so far in the Beta: 1) We suck at making MultispeQ devices quickly and 2) Most people just need a device for a short time (a month or so).  So we decided to make a lending library for MultispeQs.  Just click on this link, fill out the form, and we’ll let you know if and when we can get you a device.

Mobile Phenotyping Group

Also, we started a crew of fearless data collectors to work with partners here at Michigan State University.  We call these guys the Mobile Phenotyping Group.  We’re working with folks like Marty Chilvers on pest management in corn, soybeans, and wheat, Jim Kelly on selection of the common bean, and a whole bunch of other folks on everything from photosynthesis research to forest management.  If you are at Michigan State and you’d like our help designing, implementing, and analyzing data collected using the MultispeQ please go apply here.

Project highlights

We realize that the site is a bit messy, and sometimes it’s hard to find really interesting projects, so I thought I’d list a few to check out.  Click on ‘dashboard’ to see the raw data and play with it, or the discussion button to see what people have been saying about the project.

Nitrogen Management in CA systems in Malawi (Plant) 11. – This was data collected in Malawi comparing 5 different crop rotation systems with some surprising results (farmer recommendations on which treatment is optimal would NOT be the same for each location!) .  If you click on ‘dashboard’ and map you can see the 3 site’s where the data were collected from.

Testing Parents for Genetic Variation -Jared Crain from Kansas State University was able to identify very small (<3%) differences in photosynthetic efficiency of photosystem 2 between two elite wheat lines.

MSU’s Sonya Lawrence is having her summer biology students create projects – here’s a few:

BS 172 US15 Photosynthetic Activity – [Beta]
Tree Age influence on Photosynthetic rates. – [Beta]
Sugar Tyme Efficiency Sum 15 – [Beta]
I’m going to improve the quality of the projects and the results pages, so keep an eye out for more highlighted projects in the future.

Improvements to Forum

The website forum has been updated, and many problems are now fixed.  You can communicate with other collaborators within your project using the forum, get tech support (we trawl the tech support forum so you’ll get an answer as fast or faster than emailing me directly!) or find others with similar interests.  You can adjust your forum settings in the “Manage Subscriptions” part of your user profile page (see image below).

 

 

Inline image 1
Go to your user profile and click Manage Subscriptions to join a forum. Any projects you join or create you are automatically included in the forum.
Screenshot from 2015-06-08 08:12:35
Forums are located in the Forums tab.

v1.0 update

We are furiously working on the next version of the MultispeQ, v1.0.  We’re getting the first test PCBs out soon, and the case is being redesigned with some important improvements like integrated cuvette measurements (so no separate cuvette holder to install) and integrated leaf thickness sensor and leaf temperature.  At some point in the next 3 months, we’ll give a more significant update on the features of this next version, so stay tuned.