User and Study Data Diagnostics

Every semester, students come and students go, new researchers may arrive, new faculty might be hired, and other changes often occur beyond fresh rosters for new or updated courses. Worse still, it’s often impossible to predict important changes. Undergraduate and graduate researchers may leave one semester after they are added to your Sona site, or they might leave after several years. A study may not have a clear end date when it is added. The list goes on, but the general message is the same: your Sona site’s users and usage can be very irregular. This is why we’ve provided you with a simple, effective tool to quickly and easily track just who is and—perhaps more importantly—who is not using your Sona site. We’ve found this gem isn’t as known among Sona administrators as it should be, and we believe this is unfortunate enough to warrant a post introducing you to it. Let’s start with how to access it.

Take a minute to look over your own charts (we’ll wait here while you do), especially if this is the first time. They are filled with useful information that can prove invaluable for maintaining your Sona Systems site.

Note that we’ve broken down the data and associated charts into helpful time periods. After all, if a researcher hasn’t logged into the system in over three years, then they are probably not at your university anymore (which would explain why your emails go unanswered!). Even for most participant pools, three years is a long time. Thus, you can tell from a mere glance at the charts whether or not, for example, you have a large percentage of users whose accounts are still around even though the account holders are almost certainly not. Moreover, for precise figures, it’s a simple step to go from these visual representations to the corresponding number in the table (color-coded by column for convenience and alliteration).

Also keep in mind any recently completed administrative tasks that may influence what your data diagnostics page looks like. One common scenario would be the effect of batch imports on the first column in the table on “Last Login Date by User Type”. This gives you the number of users of each type who have never logged into your Sona site. If you’ve recently uploaded one or more course rosters using the import wizard, then you’ll have a list of users with accounts that have never been accessed. The individuals whose accounts you recently created through batch import haven’t had time to log in. In fact, they may not know that they have Sona accounts at all! At least not at first. The point is that there are processes, such as batch import, that can affect what you see when you visit your sites data diagnostics page.

For the most part, it’s not recent administrative activity that informs how you read the graphs and tables. Quite the contrary. Usually, it’s the other way around.

These tables and charts won’t just give you an accurate, quick overview on data aging in your system. They do this, of course, and do this well, but they can also directly inform administrative actions. We can make this much more concrete with a couple of examples.

Example 1: Deleting Idle Users

For the sake of this example, let’s suppose that the first of two images above (the one containing last login information by user type) was from your Sona site. Further suppose, and for the same reason, that you know the average length of time participants remain in your pool is about half a year (and never more than one year). In other words, there shouldn’t even be any participants with active accounts for three years, let alone participants who haven’t logged in for that length of time. You’ve diagnosed the problem, so what can you do to fix it?

Simplicity itself. You would just select “System Maintenance and Data Management” from the “Tasks” dropdown menu. Next, click on the “End-of-Semester Maintenance” button. This takes you to the page with all of the options for handling “idle” users. Scroll down until you see the “Idle Participant Deletion” button and corresponding textbox with a date. For the sake of illustration, let’s say you want to see all those participants who haven’t logged into their accounts in over three years vanish from your table. Simply set the deletion date to three years ago, and click the button. This will take you to a confirmation page. After making sure you set the correct date, click “Yes” to proceed. Done!

If you scroll further down on the End-of-Semester Maintenance page, you’ll note that you can do the same thing for other user account types as well. Idle researchers can be handled in the same way as participants. Just scroll to the “researcher” section of this page, and you will find an “Idle Researcher Delete” button with a corresponding textbox to set a date. Instructor accounts can also be deleted from the End-of-Semester Maintenance page.

Example 2: Deleting Old Studies

Decluttering may involve more than simply getting rid of old user accounts. Once again, though, the data diagnostics tables and graphs can help you to determine what needs to go. Studies, for example, are often left for researchers to delete when they are no longer needed. Sometimes, however, researchers may move to a new position somewhere and forget to do this, or an email notification to Sona admin could happen during a changeover and be missed, or any one of a hundreds of different scenarios could result in the same ending: A study remains on your Sona site and the researchers on the study aren’t around anymore.

Of course, just because the researchers listed on the study information page have moved elsewhere doesn’t mean that nobody at your institution is working on this project (many studies, and indeed perhaps most studies, are part of ongoing projects that involve multiple studies of differing types across various institutions and over several years). Does this mean Sona administrators should track down every researcher in a department before deleting an old study?

This brings us to an important point, and one that too often isn’t clarified (particularly for Sona administrators who aren’t themselves researchers, such as IT personnel or administrative staff). Sona servers do not store research data (the exceptions are prescreen data and online, internal studies conducted using the Sona survey tool, both which can be exported and safely stored locally). One of the reasons is precisely to make it easier to delete unneeded personal information. The Sona platform helps structure as well as optimize the research process, but data gathered during studies are kept by researchers and should be de-identified. In other words, the only “identification” researchers keep consists of a list of generic identifiers (P001, P002, P003, etc.) allowing them to distinguish participants without retaining any personal information. Deleting a long completed Sona study helps ensure no identifiable, personal data are kept that shouldn’t be. Apart from participant names, the only thing erased when you delete an old study are details related to scheduling and compensation, not data researchers collected during the study.

Deleting old studies, therefore, is part of protecting participants (rather than endangering ongoing research). And using Sona’s data diagnostics can certainly help you identify an “aged” study that needs deleting. It is not, however, where you go to actually delete old studies. This can be done easily, however, by selecting “View Studies” from the “Studies” dropdown menu. The “Batch Study Delete” button is above the study listing table:

Clicking this button will bring you to the Batch Study Deletion page. Here, you’ll find a table displaying all studies eligible for deletion. Click on “Last Activity Date” to sort the table by date. Finally, select the old, outdated studies to remove and then click “Delete Selected Studies” to delete the whole batch!

Removing “aged” studies as well as old, unused user accounts isn’t just about decluttering your Sona site. It is also an important aspect of protecting participants. This brings us to our last (but by no means least) topic on the use of your site’s data diagnostics page.

As a general rule, participant’s personal information collected during research should be kept only so long as is necessary. This is inline with, for example, the GDPR principles for data retention and erasure policies, not to mention long-standing recommendations for best practices for researchers worldwide. While actual laws and regulations vary considerably from place to place, the general guidance and recommendations here are rather universal and surprisingly clear.

By way of comparison, consider the regulations and guidelines that govern the retention and/or deletion of more general data collected during research. In some cases, there may be little to no guidance (let alone regulations) for when these data must be deleted (if ever). Indeed, in many cases researchers may be required to keep de-identified data for years, long after the study is completed. This kind of data retention doesn’t apply to sign-up sheets.

More generally, it doesn’t apply to participant contact details and similar personal information of the type that is on your Sona site. The Sona platform makes it easy to delete this personal information (including automatically de-identifying data in many cases, such as the use of anonymous participant ID codes for external online studies), in part by keeping the participant pool information separated from that collected during actual studies. Like we said: It’s not only about making sure your Sona site runs more smoothly and is free of “clutter”. It’s also about protecting participants’ privacy!

It’s good to get into the habit of taking a look at your data diagnostics page a few times a semester. That said, you’ll want to wait until the semester’s end to use your Sona site’s data diagnostics. In other words, hold off on erasing data “clutter” or deleting “aged” and unneeded studies or accounts until the semester is over. Looking at your diagnostics page during the semester will give feel for your site’s usage. It will also prepare you for end-of-the-semester maintenance (which should always be completed before you begin preparing for a new semester!).

If you haven’t already done so, please take a look at this page to see the data for your Sona site. It’s definitely worth a few moments of your time, and more than likely could end up saving you time, energy, and more even in the short run!