Data Derived From AI: Understanding The Legal Risks Of Synthetic Data, Embeddings And AI-generated Data

Artificial intelligence (“AI”) systems are increasingly generating new categories of data which do not fit neatly into existing definitions of “data” in traditional software-as-a-service (“SaaS”) contracts. In a conventional SaaS environment, parties typically distinguish between “customer data”, “usage data”, “aggregated data” and “anonymised data”. However, AI systems introduce additional categories of data such as prompts, outputs, embeddings, vector databases and synthetic datasets, which do not sit neatly within these traditional definitions.

Vendors may assume that existing anonymisation or aggregated data carve-outs are broad enough to permit them to use AI-generated or AI-derived data. Customers, on the other hand, may take the view that those carve-outs were never intended to permit the vendor to retain, commercialise or reuse embeddings, synthetic data or other AI-generated data derived from their confidential information. Regardless of whether you a customer or supplier of AI systems, it is in your interest to ensure that your agreement appropriately addresses these additional datasets.

• Synthetic and derived AI data

Synthetic data is generally understood as data created by AI systems based on learned patterns, rather than data collected directly from real-world sources. For vendors, synthetic data holds commercial value because it can provide a continuous source of training data without requiring the use of actual customer data.

If synthetic data has been generated from patterns, characteristics or insights extracted from customer information, the customer may argue that the vendor should not be entitled to use that data for its own purposes without the customer’s consent. Conversely, the vendor may argue that synthetic data is no longer customer data and should therefore fall outside the customer’s ownership and control, i.e. the vendor should be free to commercialise such synthetic data in the same way that it is typically able to commercial anonymised and aggregated data under SaaS contracts.

• Embeddings and vector databases

An embedding is a numerical representation of meaning which allows AI systems to compare, retrieve and process information. Embeddings are anonymised representations that encode numerical meaning and should not be viewed as mere statistical summaries of data. This raises important legal questions. If an embedding is generated from customer data, does it remain customer data? Can it be reused by the vendor across customers? Can it reveal commercially sensitive patterns if combined with other data?

Similarly to synthetic data, a typical SaaS clause which permits the vendor to use anonymised or aggregated data may not be suitable for embeddings or vector databases. Customers may argue that embeddings and vectors databases are derivatives of the customer’s confidential information and data and that the vendor should not be entitled to commercialise or use same beyond the ambit of the contract.

• Use of customer data for training and model improvement

Customers commonly seek express prohibitions on the use of their data for training, particularly where the data is sensitive, regulated (such as, personal information or financial information) or commercially valuable (such as, trade secrets or know-how). Robust clauses should prohibit the vendor from (i) using the customer’s data to train, retrain, or enhance their own models; and (ii) providing such customer data to third parties to train, retain or enhance third party AI models.

Vendors, however, often seek flexibility to use data to improve their models, develop new models, or to enhance existing products and services. The rationale being that as the vendor gains experience it carries these learnings through to all customers to enable them to all benefit at a reduced cost.

Privacy and confidentiality risks

Parties should be cautious about assuming that derivative data such as synthetic data, embeddings, and vector databases are automatically outside the scope of data protection law. Even where the original dataset has been altered, transformed or encoded, a privacy risk may remain if the data can identify individuals or juristic persons, or can be re-identified to such an extent so as to identify individuals or juristic persons.

If AI-derived datasets are created from a customer’s confidential information, the customer (or even the contract) may require those datasets to remain subject to confidentiality obligations. The vendor may instead seek to treat those datasets as its own technology, analytics or improvement data. It is for this reason that the contract must clearly distinguish between the different types of derivative data created or generated by AI systems and such contract must clearly indicate each party’s ownership and rights in respect of such derived data.

How can customers protect themselves?

Customers should consider implementing the following contractual protections when procuring AI systems:

• clearly define AI-related data categories, including prompts, inputs, outputs, synthetic data, derived data, embeddings and vector databases;
• state whether each category is owned by the customer, the vendor or jointly;
• prohibit the use of customer data for model training, fine-tuning or service improvement unless expressly agreed;
• require the vendor to explain how customer data is used, excluded or separated from model improvement processes;
• avoid relying solely on legacy SaaS anonymised or aggregated data clauses;
• restrict the vendor from reusing AI-derived data across customers; and
• require transparency where synthetic data is generated from customer inputs, outputs or confidential information.

Based on the above, AI-derived data is likely to become a core issue in SaaS contracts. Synthetic datasets, embeddings and vector databases are not simply technical by-products of these processes as they may carry real legal, commercial and regulatory risk. It is therefore critical that contracts address these data categories deliberately, rather than attempting to force them into legacy SaaS drafting which was never drafted to cover AI systems.