It’s important for all parties to have their incentives aligned and not feel like they’re on opposing sides.

Let’s begin by evaluating pricing options for your project. Most data vendors and clients will go for flat rate per task pricing based on the task complexity and requirements. While this makes the cost very predictable for the research team client, it can make the quality more unpredictable.

We recommend working with vendors on an hourly basis and auditing the hours of experts based on averages for each project. Your vendor should also have tools to assist in these efforts, which will be covered in a later section.

Paying experts on an hourly basis encourages perfection and double-checking for adherence to instructions.

Paying experts on a per-task basis encourages rushing and disincentivizes asking for clarification or re-reading project instructions, especially since experts can usually only work on one task at a time.

There are many cases when you, the research team, pay the data vendor on a per task basis, but in most cases, the vendor pays the experts on an hourly basis. This creates a strong incentive for the data vendors to instruct their experts to get the tasks done as fast as possible and stretch the bounds of your service level agreement for task quality. As such if you pay a data vendor a flat rate per task, you should inquire as to how the experts get paid. Everyone’s incentives can align when the data vendors charge a flat percentage fee on top of the expert’s hourly rate.

Paying hourly instead of per-task may sound more expensive, but consider again that vendors billing you per-task may take shortcuts to preserve their margins. While you have a fiscal responsibility to your stakeholders, you also have a responsibility to get high quality training data.

micro1 works with Foundational model companies on an hourly basis. We assign experts with a minimum of 10h/week per expert in most cases, and charge for the total hours they put in, once a month. We also handle global payroll, benefits, and compliance so that research teams don’t have to worry about the tedious HR aspects of scaling up a large global team.