Ajax Privacy and Cybersecurity Lawyer, Rajen Akalu, of Akalu Law P.C., provides his thoughts on regulatory guidance from the Office of the Privacy Commission in relation to artificial intelligence.

Summary

In response to developments in artificial intelligence (AI) The Office of the Privacy Commissioner of Canada (OPC), like many privacy commissioners, issued regulatory guidance for maintaining privacy protection. This paper argues that these efforts do not appropriately deal with the core problem of getting organizations’ management involved in privacy strategies that meaningfully protect personal data and individual harm.  Instead, the OPC should shift its focus to assessing for unauthorized surveillance rather than privacy harms in response to developments in AI. This shift in emphasis would reconcile the inherent tension between privacy prerogatives enacted by the current legislation and AI imperatives directed by technological and market forces.

 Introduction

In response to developments in artificial intelligence (AI) The Office of the Privacy Commissioner of Canada (OPC), like many privacy commissions issued regulatory guidance for maintaining privacy protection.  Its guidance document is titled: Principles for responsible, trustworthy and privacy-protective generative AI technologies (Canada 2023a).  The stated aim of this document is to help organizations developing, providing, or using generative AI apply key Canadian privacy principles.  The document is intended for developers and providers as well as organizations using generative AI (or individuals acting on behalf of an organization).  The principles outlined in the document focus on privacy legislation and regulation, and how they may apply to organizations (emphasis added).

While this is a useful starting point for discussion for organizations to consider privacy when developing technologies that incorporate AI, there is little substantive guidance provided.  The regulatory approach taken by the OPC in issuing guidance is common to many other privacy regulators around the world is as vacuous as it is unhelpful.  For example the EU guidelines on ethics in artificial intelligence: context and implementation (Tambiama 2019) cite privacy and data governance as a key requirement for achieving ‘trustworthy’ AI.  In the US, the Whitehouse published its Blueprint for an AI Bill of Rights which has data privacy as one of its central pillars is equally aspirational, seeking to “make life better for everyone” (Whitehouse 2022).  This paper argues that these efforts do not appropriately deal with the core problem of getting organizations’ management involved in privacy strategies that meaningfully protect privacy.  In order to more appropriately address the current privacy concerns, the OPC should shift its focus to assessing for surveillance rather than individual privacy harms. This shift in emphasis would reconcile the inherent tension between privacy prerogatives enacted by current legislation applicable the Canadian private sector and AI imperatives directed by technological and market forces.

The Office of the Privacy Commissioner of Canada (OPC)

The Office of the Privacy Commissioner of Canada (OPC) provides advice and information for individuals about protecting personal information.  They also enforce two federal privacy laws that set out the rules for how federal government institutions and certain businesses must handle personal information.[1]  The focus of this paper is the Personal Information Protection and Electronic Documents Act (PIPEDA) which regulates the collection, use and disclosure of personal information within the private sector. Personal information is defined as “information about an identifiable individual, but does not include the name, title or business address or telephone number of an employee of an organization”[2] The purpose of the Act is stated as follows:

in an era in which technology increasingly facilitates the circulation and exchange of information, rules to govern the collection, use and disclosure of personal information in a manner that recognizes the right of privacy of individuals with respect to their personal information and the need of organizations to collect, use or disclose personal information for purposes that a reasonable person would consider appropriate in the circumstances.[3]

It can be observed that there exists an inherent tension between the privacy rights of individuals and business interests. Implicit in the legislative aim is the notion that individuals ‘control’ the personal information and consent it its collection, use and disclosure.  Austin has commented that “PIPEDA suffers from a number of defects, both interpretive and structural that have their roots in these issues regarding the relationship between control and privacy.”(Austin 2006). There have been many critiques both by academics and indeed the OPC on the limitations of over-reliance of consent in the protection of individual privacy (Solove 2013; Canada 2018).

Canada’s decision to enact a self-regulatory code would invariably lead to problems of enforcement since industry codes are policy tools of soft compliance.  Stevens argues that Canadian Standards Association Model Code, which is Schedule to PIPEDA, though hailed as a product of consensus reflected a very limited set of interests in efficiency and economic gain.

Had the government based PIPEDA on its peace, order, and good government power instead of its trade and commerce power, privacy could have been prioritized as a human right.  The decision to afford primacy to economic growth and efficiency has the effect of positioning individuals first as consumers.  This she writes has far-reaching implications for the current privacy reforms contemplated for AI systems (Stevens 2024). The debate is framed as one of individual privacy verses legitimate business interests.  In such a discussion privacy rights will be regarded as negotiable rather than inalienable.

Nevertheless, the OPC in its public statements seeks to articulate privacy as a human right.  In outlining the privacy rights and the substantive goals of privacy regulation, the Privacy Commissioner of Canada, Phillippe Dufresne, declared privacy to be a fundamental right (Canada 2023b). In his address to the Canadian Bar Association he cited the 2014 Supreme Court of Canada case United Food and Commercial Workers, where it was held that “legislation which aims to protect control over personal information should be characterized as ‘quasi-constitutional’ because of the fundamental role privacy plays in the preservation of a free and democratic society.”[4] He also expressed that the quasi-constitutional nature of privacy has significant regulatory implications in the following way:

It means that privacy must be legally protected and promoted with a strong, fair and enforceable legal and rights-based regime. A regime that must offer meaningful remedies that prevent and address violations and that will act as an incentive for institutions to create a culture of privacy…Privacy by design where it is considered, valued, and prioritized. Privacy that is included and embedded at the outset of innovation, not as an afterthought or regulatory irritant (Canada 2022)

The decision to adopt a rights-based framework based on constitutional protections with a view to creating a ‘culture’ of privacy provides individuals with weak basis to assert their rights.

The rhetoric of regulators and the requirements of private organizations as to the appropriate emphasis of privacy law and policy are likely to differ on whether privacy regulation should adopt a rights-based framework.  Unless the organization is in the business of protecting privacy, that business is far more likely to view privacy as a variable that must be controlled by meeting regulatory compliance obligations than it is to conceptualize it as a ‘quasi-constitutional’ right that cannot be negotiated. While there may not be consensus on which approach will work best for the purpose of optimizing the strategic use, quality, and long-term availability of personal data, it is widely acknowledged that generative AI pose novel privacy harms that will need to be addressed (Gupta et al. 2023). This is because generative AI require the processing copious amounts of potentially sensitive information. This data is obtained or ‘scrapped’ from publically available internet sources. Generative AI, is a set of AI approaches that uses a machine learning model to produce new material including text, pictures, audio, and video (Bale et al. 2024).  This is to be contrasted with predictive modelling where algorithms can analyze seemingly unrelated factors such as purchase history, salary, family relationships to make deductions about an individual and machine learning which is used to create predictions, automate procedures and make recommendations (Hermann and Puntoni 2024). This data is typically generated by an individuals’ interaction with an organization. Though not made explicit in the OPC guidance generative AI technologies represent a greater challenge to privacy law since the data is typically obtained from publically available sources.  The reliance on individual consent to share personal information which is a core tenet of privacy law is simply unworkable under these circumstances. This is becoming increasingly apparent in attempts at regulatory reform in this area.

The OPC Guidance

On December 7, 2023 the OPC issued its guidance Principles for responsible, trustworthy and privacy-protective generative AI technologies.  The Commission claims its authority to regulate in the field of AI with respect to privacy by making the following statement.

While generative AI tools may pose novel risks to privacy and raise new questions and concerns about the collection, use and disclosure of personal information, they do not occupy a space outside of current legislative frameworks. Organizations developing, providing, or using generative AI are obligated to ensure that their activities comply with applicable privacy laws and regulations in Canada. Organizations should also recognize that to build and maintain a digital society in which innovation is socially-beneficial and human dignity is protected, AI development and use have to be responsible and trustworthy (Canada 2023a).

It is well established that the Commissions’ regulatory fiat is implicated in the context of commercial activity where personal information is involved.  The guidance goes on to explain how the data handling principles of limiting collection, accountability, individual access, safeguards etc. apply to AI developers and providers as well as organizations using AI.

However, it is relatively easy to generate regulatory guidance based on generic data handling principles.   The rationale for this approach loosely stated is that by considering privacy at the outset, privacy will be protected and trust will be established resulting in a win for individual privacy and a win for organizations.  From a regulatory perspective, this approach is highly favourable; it allows Commission to pontificate on what organizations should be doing to protect privacy as well as judge them for departing from the ideal through its enforcement function.

But as Spiekermann notes even when both managers and engineers are committed to privacy by design, there are more challenges must be overcome.  Privacy is an ambiguous concept and is thus difficult to protect. We need to come to terms about what it is we want to protect. Moreover, conceptually and methodologically, privacy is often confounded with security.  There is no agreed-upon methodology that supports the systematic engineering of privacy into systems. System development life cycles rarely leave room for privacy considerations and there is little knowledge exists about the tangible and intangible benefits and risks associated with companies’ privacy practices (Spiekermann 2012).

The problem of getting management to execute a privacy strategy at the process design stage is further complicated by the fact that AI “blurs the line between data collection and data processing, allowing end-runs around many privacy law protections”(Solove 2024).  This coupled with the overreliance on consent and facilitated by notice and choice approaches, creates a situation where “businesses can do what they want with user information provided (1) they tell users they are going to do it  and (2) users choose to proceed” (Susser 2019). Moreover, the power of AI and its ability to make inferences from large data-sets of publically available information (e.g. scraping – the non-consensual gathering of data online) obviates the need for consent in most cases.

The ambiguity regarding the fundamental privacy rights that require protection coupled with vested economic and business interests is resulting in what Walden has observed as the managerialization of privacy law. Walden’s critique of modern privacy law regimes concludes that:

Privacy law is failing to deliver its promised protections in part because the responsibility for fulfilling legal obligations is being outsourced to engineers who see privacy law through a corporate, rather than a substantive, lens (Waldman 2019).

The ambiguity in legislative requirements permits privacy lawyers, technologists and other privacy compliance professionals to determine what privacy law means in practice. This being the case, the law will be interpreted to align with managerial values such as risk mitigation, operational efficiency, and maintaining consumer trust rather than the substantive goals that privacy law is meant to facilitate namely, privacy protection, individual liberty and democracy.

In an attempt to provide industry direction to companies seeking to comply with privacy legislation, the OPC issues regulatory guidance.  However no serious attempt by the OPC is made to engage in the problem inherent in almost all generative AI use – that generative AI use runs contrary to practically all privacy protection principles.  Without a fundamental shift away from privacy protection, regulatory guidance is likely continue to be both weak, unenforceable and misguided.

Toward an alternate model of safeguarding against privacy harms

The trade-off between the privacy rights of individuals and business interests is reconciled by defining personal information and making that subject to data handling principles enshrined in legislation.  Individual consent regulates the sharing of personal information in this model. But since individuals are unable to assess the risk associated with the sharing of personal information, compliance with privacy law outcomes will be biased in favour of managerial prerogatives such as efficiency and risk mitigation.  However, privacy is a social value the commoditization of which results in its loss.  This approach will invariably result in more privacy regulation and less privacy.

The shortcomings of the current approach are brought into focus with the use of AI.  Cohen argues that contemporary practices of personal information processing constitute a new type of public domain.  She refers to this as the biopolitical public domain which is described as “[a] repository of raw materials that are there for the taking and that are framed as inputs to particular types of productive activity.” (Cohen 2018)  The term biopolitical underscores the fact that the information either identifies or relates to people.

Solove and Hartzog have similarly observed that AI systems depend on massive quantities of data, often gathered by “scraping” – the automated extraction of large amounts of data from the internet.  This processed data becomes a source of competitive advantage and valued as such.  The regulatory problem to solve is not defining personal information but rather the use of publically available data that is able to make human behaviour and preferences predictable and profitable in aggregate (Solove and Hartzog 2024).  The individual, so central to the current privacy regulatory model is placed with a probabilistic gradient that escapes meaningful oversight.

The business practice of scraping data for use by AI systems can be viewed as a form of commercial surveillance activity. Lyon defined surveillance as “the focused, systematic and routine attention to personal details for purposes of influence, management, protection or direction.”(Lyon 2001). The gathering of data for AI systems used by private companies fits Lyon’s definition squarely.  Solove and Hartzog argue in favour of framing the issue as one of data security such that “[f]ailing to implement reasonable protections against scraping by third parties is tantamount to improperly sharing data with third parties” (Solove and Hartzog 2024).

However, they also acknowledge that not all scraping results in harm.  Indeed there are many beneficial uses of scraping and surveillance activities ranging from health care to scientific and historical research.  They instead argue in favour of a public interest requirement as opposed to individual consent with respect to collection, use and transfer of personal data use by AI systems.  This they argue would make scraping a privilege by conditioning it in justified contexts upon the adoption of safeguards and commitments that benefit society as a whole.  However, determining whether commercial scraping is in the public interest is likely to result in inconsistencies in application as the regulators would be required to have near-omniscient knowledge of the potential misuse of data.  In the EU the GDPR opted for a consent and legitimate interests approach as the lawful basis of scraping personal data by the private sector.[5] This approach is likely to suffer from the same problems experienced by practically all consent-based privacy regimes, namely that individuals are in no position to assess the risk associated with information sharing and commercial organizations are deft at asserting that their interests in collection use and disclosure are legitimate.

Given the potential for harm it is reasonable for scrapers to be regulated by privacy law when they collect and use personal data.  However, this is a complex policy problem and the regulatory response to it should be nuanced.  An outright ban on scraping would limit socially beneficial uses, regulating based on the ‘public interest’ is ambiguous and will be subject to the discretion of the regulatory agencies.

 Guidance on what constitutes commercial surveillance and when greater transparency will be required

Providing guidance on the causal relationships generated by algorithms is important because AI focuses on correlative relationships and interpretability varies with a model’s complexity.  Interpreting the outcome of a linear regression model for example is easier than predictions made by a neural network.  As the number of parameters increase so too will the model’s inscrutability.   This complicates a problem central to modern privacy regulation, namely that as long as “users remain the product, there is minimal incentive for these companies to provide any real privacy”(Schneier 2013).

The regulatory concepts of legitimate interests and consent are too malleable in the hands of private organizations to provide adequate privacy protection.  Since contemporary privacy law aims at protecting harm to individuals, businesses will necessarily respond by developing increasingly sophisticated privacy and AI governance frameworks in order to demonstrate compliance with privacy law.  An alternative option with the potential to reconcile these policy reform orientations is to creatively rethink the obligations of organizations and the practices of transparency and accountability with respect to profiling (Austin 2014).  Transparency, like consent and purpose specification, is a requirement of PIPEDA and a mainstay of fair information practices.  Pursuant to the openness principle PIPEDA states that detailed personal information management practices must be clear and easy to understand. Shifting the emphasis from authority collect use and disclose personal information based on individual consent toward legal authority to engage in commercial surveillance is where guidance would be of greatest benefit.  Establishing guidance on when commercial surveillance is unauthorized would bring much needed clarity to businesses seeking to use and develop responsible AI technologies.

Conclusion

The current approach to regulatory guidance by the OPC and other privacy regulatory authorities is unlikely to provide meaningful direction to organizations, even if those organizations are sincerely committed to developing responsible AI systems.  This is because private organizations will invariably view the privacy problem to be solved in terms of efficiency and economic gain.   Individual consent is a necessary but insufficient approach to privacy protection with respect to AI technologies when balanced vis-à-vis legitimate business interests.  The guidance issued on Responsible AI is based on a self-regulatory code that seeks to promote best business practices for data protection where individuals are ostensibly afforded control of their personal information.  This paper argued that the OPC should instead shift its focus to assessing for unauthorized surveillance in the private sector rather than attempting to identify individual privacy harms in response to development in AI. This shift in emphasis would reconcile the inherent tension between privacy prerogatives enacted by the current legislation and AI imperatives directed by technological and market forces.

 References 

Austin, Lisa M. 2006. “Reviewing Pipeda: Control, Privacy and the Limits of Fair Information Practices.” Canadian Business Law Journal 44:21.

———. 2014. “Enough About Me: Why Privacy Is About Power, Not Consent (or Harm).” SSRN Scholarly Paper. Rochester, NY. https://papers.ssrn.com/abstract=2524512.

Bale, Ajay Sudhir, R. B. Dhumale, Nimisha Beri, Melanie Lourens, Raj A. Varma, Vinod Kumar, Sanjay Sanamdikar, and Mamta B. Savadatti. 2024. “The Impact of Generative Content on Individuals Privacy and Ethical Concerns.” International Journal of Intelligent Systems and Applications in Engineering 12 (1s): 697–703.

Canada, Office of the Privacy Commissioner of. 2018. “Guidelines for Obtaining Meaningful Consent.” May 24, 2018. https://www.priv.gc.ca/en/privacy-topics/collecting-personal-information/consent/gl_omc_201805/.

———. 2022. “A Vision for Privacy: Rights, Trust and Public Interest.” November 10, 2022. https://www.priv.gc.ca/en/opc-news/speeches/2022/sp-d_20221104/.

———. 2023a. “Principles for Responsible, Trustworthy and Privacy-Protective Generative AI Technologies.” December 7, 2023. https://www.priv.gc.ca/en/privacy-topics/technology/artificial-intelligence/gd_principles_ai/#fn10-rf.

———. 2023b. “Privacy as a Fundamental Right in the Digital Age.” March 10, 2023. https://www.priv.gc.ca/en/opc-news/speeches/2023/sp-d_20230224/.

Cohen, Julie. 2018. “The Biopolitical Public Domain: The Legal Construction of the Surveillance Economy.” Philosophy & Technology 31 (2): 213–33. https://doi.org/10.1007/s13347-017-0258-2.

Gupta, Maanak, Charankumar Akiri, Kshitiz Aryal, Eli Parker, and Lopamudra Praharaj. 2023. “From ChatGPT to ThreatGPT: Impact of Generative AI in Cybersecurity and Privacy.” IEEE Access 11:80218–45. https://doi.org/10.1109/ACCESS.2023.3300381.

Hermann, Erik, and Stefano Puntoni. 2024. “Artificial Intelligence and Consumer Behavior: From Predictive to Generative AI.” Journal of Business Research 180 (July):114720. https://doi.org/10.1016/j.jbusres.2024.114720.

Lyon, David. 2001. Surveillance Society. McGraw-Hill Education (UK).

Schneier, Bruce. 2013. “‘Stalker Economy’ Here to Stay.” CNN. November 20, 2013. https://www.cnn.com/2013/11/20/opinion/schneier-stalker-economy/index.html.

Solove, Daniel J. 2013. “Introduction: Privacy Self-Management and the Consent Dilemma.” Harvard Law Review. May 20, 2013. https://harvardlawreview.org/print/vol-126/introduction-privacy-self-management-and-the-consent-dilemma/.

———. 2024. “Artificial Intelligence and Privacy.” SSRN Scholarly Paper. Rochester, NY. https://doi.org/10.2139/ssrn.4713111.

Solove, Daniel J., and Woodrow Hartzog. 2024. “The Great Scrape: The Clash Between Scraping and Privacy.” SSRN Scholarly Paper. Rochester, NY. https://doi.org/10.2139/ssrn.4884485.

Spiekermann, Sarah. 2012. “The Challenges of Privacy by Design | Communications of the ACM.” 2012. https://dl.acm.org/doi/10.1145/2209249.2209263.

Stevens, Yuan Y. 2024. “Model Standards, Model Law? The Path Dependence of PIPEDA’s Enforceability Issues.” https://doi.org/10.13140/RG.2.2.21257.04960.

Susser, Daniel. 2019. “Notice After Notice-and-Consent: Why Privacy Disclosures Are Valuable Even If Consent Frameworks Aren’t.” Journal of Information Policy 9 (December):148–73. https://doi.org/10.5325/jinfopoli.9.2019.0148.

Tambiama, MADIEGA. 2019. “EU Guidelines on Ethics in Artificial Intelligence: Context and Implementation.”

Waldman, Ari Ezra. 2019. “Privacy Law’s False Promise.” Washington University Law Review 97:773.

Whitehouse. 2022. “Blueprint for an AI Bill of Rights.” https://www.whitehouse.gov/wp-content/uploads/2022/10/Blueprint-for-an-AI-Bill-of-Rights.pdf.

[1] Personal Information Protection and Electronic Documents Act (S.C. 2000, c. 5) (PIPEDA) and Privacy Act (R.S.C., 1985, c. P-21).

[2] Ibid s. 2.

[3] s. 3 PIPEDA.

[4] United Food and Commercial Workers, Local 503 v. Wal‑Mart Canada Corp [2014] 2 SCR 323.

[5] GDPR, Article 5(1)(b).