Compared to movisens
First Impression: Focused on Sensors
Although they say they support collecting both subjective and objective data, some clues show movisens is more focused on their sensors instead of experience sampling. For example:
- On their Products page, movisensXS is among the last items while their sensors are presented at the top.
- On their Documentation page, again, the Sensors and Data Analysis Software sections are before the movisensXS section.
- They have more advanced features for analyzing and working with data from their offered sensors, but their sampling data lacks such features. You get a simple chart based on the number of completed forms and for more details, you should download data.
- The title "movisens" seems to be based on "sensors" while "movisensXS" seems to be an addition.
Too Many On-Request Features
One of the first things you'll notice after checking their documentation is the number of features that are labeled "On request". For such features you need to contact their support team; you can't even test them using the dashboard by default.
For example, they have a Visual Analog question type, but if you want to have "Image Extremes" for your VAS, you should use another question type and that's "On request".
We decided not to list all such features and properties, but at the time of writing this article, based on a simple search on their documentation pages, there are at least 300 such features or properties.
movisens might have a reason behind this, but as an end user, this isn't an easy way to evaluate a system before using it. Note that based on their Pricing page, it seems that on-request features don't have extra costs.
Limited To Android
When it comes to user base, the wider range you support, the better. This doesn't seem to be the case with movisens. They decided to focus on Android only, because:
Experience Sampling has lots of requirements and building a solid solution for multiple operating systems is very difficult. The iPhone is also too restricted for research purposes. We focus on Android because cheap smartphones are available. Android is also open source and it is sustainable.
Avicenna iOS apps can be considered as counter-examples. The restrictions by Apple have meanings behind them and Avicenna tries to respect them while giving the flexibility to the participants to use whatever device they're comfortable with.
On the other hand, movisens doesn't have a web app for participants but it's cheaper to maintain (if you don't want to collect sensor data) and more accessible.
Revisiting User Experience Might Help
movisens could've benefited from some improvements in their user experience. We list some of the most important and obvious items below:
After a study is changed, there might be problems with downloading the results of participants from older versions of the study. To come around such problems, you're able to download the data from old study versions using URL flags (or query params). This doesn't seem to be a good intuitive solution.
In exported results from forms, they refer to forms and triggers by their names (not IDs) which can change. Maybe that's why exporting data from older versions is unintuitive (as mentioned above). Note that they don't offer a comprehensive Audit Trail either.
No way to manage data through the dashboard. You can download it only. They have one page for Results which includes one simple chart.
Their Audio question doesn't have a UI for the recorded/recording audio. It just says "Don't close the question" when the audio is being recorded.
Their chat system is not real-time on the researcher's side. You should refresh the page.
It seems their Sampling page has some validation checks too. But you can't see the errors. It just changes the node border colors. You should guess the errors.
You can have mutable values (i.e. variables) based on Time and Date & Time responses, but not responses to Date questions.
If you change only the layout of the sampling flowchart (for example, just drag/drop a node), it'd result in a new protocol version and needs re-consent on the participant's side.
You only receive (chat) messages from participants if you have "write" access to the study.
They have a Reaction Time form item. As a best practice, they say:
Please do not configure a headline, because the Reaction Time Item is hidden for the participant.
But it's not clear why such a form item has a headline in the first place.
They don't have any email verification after signing up. But you can, for example, reset your password.
Hard Time With Maintenance?
They say app updates may break your study in the field. So, they suggest disabling auto updates in Android.
On the other hand, you can change the library version for your study, but they recommend only doing so when their support tells you to.
These two specific behaviors don't seem to be aligned with continuous updates/improvements which is a best practice in software development. In Avicenna, we have weekly updates to make sure our users have the most fluent experience while benefiting from new features.
Also, if you check their Recent Changes and News, the frequency of their updates is low and the last update might be months ago.
Not Focusing On Personal Devices
It seems their main focus is on devices researchers give to participants. Some possible clues:
- They only support Android devices.
- They have a section on their documentation called "Prepare for a Lost or Stolen Device".
- They mentioned some requirements to "Configure a study smartphone".
- Also, the "Play Video" question needs the video to be in the smartphone's SD Card.
- The "Record Audio" feature which is part of their sampling, records the audio of the microphone for the specified duration and stores it on the smartphone's SD card.
- They offer a "Log Node Status and Variables" (which is an on-request feature
of course) for the Study Running sampling node. If that property is enabled,
node status changes will be logged into a Unisens file format under
/sdcard/movisensXS/logs/. They haven't mentioned how those logs will be accessible to the researchers though.
Registration Flow: A Double-Edged Sword
movisens participants only need a QR code from the researchers and they can start participating in no time. This has some pros and cons. Let's start with the pros:
- It's more convenient for the participant.
- Less footprint. It's like you came in, did the job, finished the study, and no one knows who you are (except probably some of your metadata like IP).
- More privacy. Participants don't need to use any authentication with email address, phone number, username, and password.
That being said these cons might come to mind:
As a researcher, you can't use email and SMS notifications to inform participants.
In some research studies, knowing about the PII of participants is necessary. movisens doesn't offer that and it seems they even try to prevent it. In their FAQ page, they say:
movisens does not know the identity of the participant. All participants get a pseudonym. In the study no personal data is allowed to be collected that allows to identify the participant (e.g. name).
movisens Strength
movisens' Sampling page is very intuitive and easy to use. Sampling is done by moving around some flowchart nodes, connecting them with some logical restrictions, and adjusting the properties in each node. With this user interface, researchers also have a complete overview of what should happen for participants.
We haven't tested this for bigger studies with very complex sampling protocols. Although we know that big/complex flowcharts are not easy to traverse and digest.
On the other hand, for movisens Forms, you can only define internal protocol and branchings. But it's the sampling that makes those forms available for the participants. So, we see this separation (which is not 100% perfect since it has internal branchings in forms) as a nice thing to have.
Feature Category Comparison
The following table compares Avicenna and movisens on different categories of features:
| Category | Superior |
|---|---|
| Study Setup & Deployment |  |
| movisens lacks features in many areas like localized study content, eligibility screening, enrollment available to the general public, participant management, Televisit (Chat), and multi-site studies. Also, they don't have any Audit Trail which is one of the most important items in an EMA system toolbox. | |
| Notifications | |
| movisens only supports in-app notifications. They don't support other mediums like email or SMS. Also, you can't define templates for notifications. And, you can't monitor the notifications and their statuses, especially notifications in the future. | |
| Participant Activities | |
| While movisens offers more features on scripting, variables, and triggering logics, they lack more complex features for managing activities and sessions. That's why Avicenna ranks a bit higher in this category. | |
| Survey | |
| movisens has some good features like the Likert questions or section-based layout for participants. But Avicenna is more flexible and feature-rich in general capabilities of surveys, question types and their properties, eCRFs, public surveys, and analyzing responses. | |
| Interventions & Cognitive Tasks | |
| Although neither movisens nor Avicenna is feature-rich in this category, Avicenna offers more cognitive tasks like Stroop and Matrix Reasoning. | |
| Gamification & Rewarding |  |
| In this category, movisens is the winner since Avicenna doesn't offer any feature. Note that movisens only supports one feature from this category. | |
| Software Security & Reliability | |
| Here, Avicenna seems to have valued security more than movisens. Avicenna supports almost three times more features including flexible study roles and permissions, complying with guidelines and regulations, and following software development's best practices. | |
| Sensor Data & Wearables | |
| While movisens offers their own sensors, we didn't include them in the comparison since they're not identified as standard ways other EMA systems should use to collect sensor data. Considering this statement, Avicenna supports more data sources including mobile sensors and wearables which are more accessible to participants. | |
| Participant app | |
| movisens allows participants to configure their apps a bit, but it lacks some bigger features like supporting iOS and web apps, dropout surveys, and participating in multiple studies in parallel. | |
| Data Access & Analysis | |
| Avicenna wins this one. It has a comprehensive data filtering system researchers can apply to almost all types of data. Also, Avicenna uses Kibana which is a powerful tool to visualize data with different types of charts. | |
| Other | Tie |
| movisens offers a good interface for monitoring the quota. But to the best of our knowledge, they do not offer custom on-premise deployments. | |
For details, please check the comparison spreadsheet.