Using AI tools at work without breaking the Privacy Act

The Privacy Act is technology-neutral. The same principles apply whether personal information goes into a filing cabinet, a spreadsheet or an AI chatbot. What’s different about AI tools is how easily staff can put large amounts of personal information into them, often through personal accounts the organisation doesn’t know about.

What the Commissioner expects

The Privacy Commissioner’s guidance on generative AI expects organisations considering an AI tool to, among other things:

• have senior leadership approval, based on the risks and benefits
• review whether the tool is necessary and proportionate
• do a privacy impact assessment before using it
• be transparent with people about how their information is used
• engage with Māori about potential risks and impacts to the taonga of their information
• develop procedures about accuracy and access by individuals
• ensure human review before acting on AI outputs, because these tools can produce confident errors and reproduce bias
• make sure personal information isn’t retained or disclosed by the AI tool in ways it shouldn’t be

The Commissioner’s general advice is to avoid entering personal or confidential information into AI tools unless you’re confident about how the provider treats it.

The question that decides most cases

Does the provider use what you put in for its own purposes?

If an AI service only processes your information on your behalf, under contract, and doesn’t keep it or train on it, section 11 of the Act treats it like any other supplier holding data for you. You’re responsible, but it’s manageable.

If the provider can use your inputs to train its models or for its own analytics, putting personal information in is a disclosure. You’d need a basis under IPP 11, and IPP 12 if the provider is overseas. In most ordinary business situations you won’t have one.

Free and personal accounts are much more likely to fall into the second category. Business or enterprise plans usually offer terms that don’t train on your data. Check the contract terms themselves.

Rules you can adopt this week

1. Use only approved tools, through organisation accounts, with settings that stop your data being used for training.
2. No personal information in unapproved tools. That includes names, contact details, health information, case notes and anything about staff.
3. De-identify where you can. Replace names and identifiers before asking for help with a document. Remember that removing a name doesn’t always make someone unidentifiable.
4. A human checks before it counts. Never send, publish or decide anything about a person based on AI output without a person reviewing it.
5. No AI decisions about people without a privacy impact assessment. Recruitment screening, credit or eligibility decisions, fraud flags and performance monitoring all need one.
6. Be honest with people. If AI is used in a way that affects them, say so in your privacy statement.
7. Know where it goes. Record which AI tools are approved, what data they can receive, and where the provider processes it.

Our AI use policy template turns these into a one-page policy.

Access and correction still apply

If you use AI output about a person (a summary, a score, a draft decision) and keep it, it’s personal information. They can ask for it, and ask you to correct it. Make sure you can find it.

Māori data

The Commissioner’s guidance specifically asks agencies to consider the impact on Māori and to engage on it. Training or tuning AI on information about Māori, or using AI in decisions that affect Māori communities, raises questions of data sovereignty as well as privacy. See Māori data and privacy.