Buying and using AI products under the Privacy Act: the OAIC's due-diligence and APP checklist

What it is. In October 2024 the OAIC published guidance for organisations that deploy commercially available AI products, such as chatbots, content generators and productivity assistants, that handle personal information. Its starting point is simple: the Privacy Act applies to all uses of AI involving personal information, and obligations attach both to the data you put into an AI system and to what the system produces. This page turns that guidance into a working checklist for security, privacy and procurement teams. It follows the OAIC page and is not legal advice.

Before you buy: due diligence

The OAIC expects you to work out what you will use the product for and whether it suits that purpose, including whether it has been tested for that use. Uses that could have a legal or similarly significant effect on someone's rights are treated as high privacy risk, and in that case you should also be able to understand and explain how the system behaves. To meet the APP 10 accuracy duty, find out what data the product was trained on and whether that data is diverse, relevant and reliable enough for you; a model not trained on Australian data may not be accurate enough for Australian uses. Developer safeguards, such as filters that stop personal information surfacing in output, will not be foolproof, so you still need your own controls. The OAIC suggests a Privacy Impact Assessment to work through these questions.

Security: APP 11 applies to the tool too

APP 11 requires reasonable steps to protect the personal information you hold, and the OAIC says you should make sure an AI product does not add security risk to your holdings. Its suggested checks: search for past security incidents involving the product; find out what the developer does to detect and block attacks, for example whether a model tuned on your data blocks queries designed to make it repeat its training data; decide where it will run (on your premises or in the cloud, and if the cloud, where the servers are and whether information could leave Australia, which brings in APP 8); and, for AI coding assistants that may touch personal information, run a risk analysis, because generated code often carries vulnerabilities and the assistants themselves can be attacked.

Read the terms: who else sees your data

The OAIC says to check whether the service terms give the developer access to the data you input or generate. Some products let the provider collect customer inputs to train further models, which risks your information surfacing in answers to other users, and some features pass data to third parties such as search engines. If the product discloses personal information to the developer or others, decide whether to use it at all and, if you do, put controls around it: turn off features that disclose data, give any notice required, stop personal information being entered, or forbid staff from entering it.

Prompts: input can be a use or a disclosure

Putting personal information into an AI system is a use or a disclosure that must comply with APP 6. A proprietary system that keeps inputs inside your organisation is a use; a public chatbot whose owner receives the input is a disclosure. Either way it must match the primary purpose for which you collected the information, or fit an exception. The OAIC reads purposes narrowly where there is doubt, and says that given community concern about AI it will often be hard to show that a secondary AI-related purpose, such as training a model, was within people's reasonable expectations. If you cannot show that, it says to seek consent or offer a meaningful opt-out. Use only the minimum personal information needed. Once information is in a generative AI product it may be very hard to track or impossible to remove, so the OAIC's best-practice recommendation is not to enter personal information, especially sensitive information, into publicly available AI chatbots.

Outputs: generating information is collecting it

The OAIC treats personal information that an AI system generates or infers as a collection under APP 3: it must be reasonably necessary for your functions or activities and obtained by lawful and fair means. Inferred, wrong or artificially generated material about an identifiable person, including hallucinations and deepfakes, is still personal information and must be handled under the APPs. Sensitive information generally needs consent, and the OAIC says consent cannot be implied just because someone was told about the collection. For accuracy under APP 10, it points to disclaimers that explain the system's accuracy limits and tools such as watermarks.

Transparency and ongoing review

Update your privacy policy and collection notices with clear information about how you use AI, and make sure public-facing AI tools such as chatbots are clearly identified as AI to the people using them. Due diligence is not 'set and forget': the OAIC expects regular reviews of the product's performance, staff training and monitoring across the product's whole life. It also points to the National AI Centre's Voluntary AI Safety Standard, a set of 10 voluntary guardrails, as a companion to the privacy obligations.

Practitioner checklist

1. Keep an inventory of AI products in use, including staff use of free public chatbots, and record what personal information each one touches. 2. For each product, record the intended use, whether it is high privacy risk, and the training-data and accuracy evidence you relied on; run a PIA for higher-risk uses. 3. Read the terms for training on customer data and third-party data flows; turn off what you can and document the rest. 4. Treat the vendor as part of your supply chain: incident history, attack protections, hosting location and APP 8 position. 5. Set a staff rule for prompts: no personal or sensitive information in public AI tools, and minimum necessary data in approved ones. 6. Treat AI output about people as collected personal information, with accuracy checks and disclaimers. 7. Label chatbots as AI, update the privacy policy, and schedule reviews rather than signing off once.

Desk note: this page summarises the OAIC's 'Guidance on privacy and the use of commercially available AI products' (published 21 October 2024, fetched 27 October 2026). It is not legal advice. The Privacy Act 1988 is the authority.