What is the best way to evaluate if an ELN solution is working for you? This is one of my favourite topics because it gets to the core of why we are trying to make the best software we can for labs. We want software that helps our clients carry out their research, so I believe it is important to be able to assess this accurately. It can be quite a difficult thing to do however, and users who are encountering a software solution for the first time can struggle to get to the root of what is really important for their organization. This is why I would like to discuss some key factors to think about when you are evaluating an ELN.

Usually, although not always, when users come to us looking for an ELN, they have some key processes in their lab for which they feel their current methods aren’t working. Maybe it is tracking the lineage of their samples, or exploring their data efficiently, or being able to see an audit trail of changes to their data. Whatever they may be, these processes which are currently a cause of friction in your organization are the key things to be focusing on. Coming up with these sorts of use cases is probably the easier task for most organizations assessing an ELN. Narrowing down the correct ones to focus on is probably more challenging. If this is the case, these are things I recommend to help this process:

• Keep it specific. Language is very important here. For example, “We struggle to compare data” tells part of a story and suggests an area where a new software could be useful but it’s not specific enough to enable you to evaluate if that new software tool can bring any tangible benefit. Conversely, “when we have run the same protocol across 100 different strains, we struggle to bring that data together so we can effectively analyse it” explains that the key issue is bringing data collected separately into one place so that analysis is possible. Having the issue stated clearly allows it to be evaluated.
• Know what you can fit into your time frame. Understanding whether your software solution will work for you within a reasonable time frame is key. We want to avoid wasted effort on long, drawn-out assessments. For this reason, I recommend picking a handful of very specific key success criteria and focusing on these. This can be hard to do when you have a number of users with different needs but eachsuccess criterion may require a fair amount of work to adequately carry out the assessment, so it is important that there is capacity to give each criterion the attention it requires. For example, if your criterion is “for audit trail purposes, I need to understand how and when data has been changed, and ensure I can access this information when required”, you may want to put a reasonable amount of effort into generating and recording data in the ELN so you can assess this thoroughly.
• Is this process going to be relevant to the work you are doing ? Where possible, try to focus on processes that are going to be use for a while in your organization. In fast moving R&D environments, experiment workflows or processes can change quickly, so try to chose something that is going to span different projects. For example, the ability to organise your cell lines and attach all required metadata to them may be more likely to continue to be relevant from one project to another within an organization than perhaps more a very specific experimental workflow.

• Within the scope of implementing an ELN tool, finding metrics that have real meaning can be problematic. Finding out how many data points have been collected on your ELN may make for an interesting figure, but what does it say about the research that has been conducted or the experience of the users carrying it out? Figures for daily active users can provide some information about the tool’s adoption, but this should be referenced with the context of the current research activities. Some things may simply be more important than numbers alone, for example, if all your users are reporting that a tool is easy to use and provides an intuitive interface, this is likely to be more relevant than whether particular users have logged in on a particular day.
• It is common to start with a small group of users to test an ELN for a period before expanding its usage across the organization. When it is only a select group of researchers employing the tool, it simply might not be possible to generate enough data, particularly over a short period, for the numbers generated to be meaningful.

I’m not advocating for ignoring metrics entirely - there may be times when an assessment based on numbers is what is needed, but care must be taken to make sure that these numbers are being looked at because they accurately reflect your end goals and not because they are something that is easy to measure.

There’s a rather easy trap to fall into when assessing an ELN, particularly as part of a short proof of concept project, which is to focus on solely on the very immediate needs of data capture and data visualization. These things are important and need to work well for the end users for adoption of the tool to be a positive process. In practice however, these are often not the things that have lead an organization to be considering implementing an ELN. Bench scientists will generally have methods very well adapted for capturing their R&D data - often this involves well formatted spreadsheets and Google Docs, and as a consequence, this aspect of carrying out research is not normally the sticking point. More often, organizations are looking for tools that allow them to record, explore and revisit their data and their workflows from multiple time points and angles. The sorts of questions which can be difficult to answer with older “pen and paper” methods but can be made easier by using an ELN are:

• What exact version of a protocol was used in an experiment?
• What was the exact lineage of a sample before an analysis carried out on it?
• When exactly was data captured? Has it been edited since then? If so, by whom?

Organizations are often drawn to an ELN because they want to make sure they are capturing the right data for filing patents; or to ensure that their data is structured effectively; or to know that their R&D managers are able to get the overview of the research when needed. When these things are the driving force behind seeking an ELN, they need to form part of the assessment process.

Assessing an ELN isn’t always going to be easy, there’s unlikely to be hard and fast metrics which are going to define whether a solution is going to work for you or not. But there are benefits in taking a defined, qualitative approach to understanding your needs and how well the software tools meets them. Being mindful of what started you on the journey to implementing or changing you ELN is key. By being very specific about the problems that your organization currently faces and which users are facing them, you can understand your own requirements and take a considered approach to ELN assessment.