Hindsight is 20/20: Assessing outcomes from Aarogya Setu

Donate to help sustain our work

Tl;dr

We are close to a year since the launch of Arogya Setu. This provides adequate time to reflect and examine its efficacy. What we have found is worrying. There is little, to no public data nor evidence on it's tangible benefits. On the contrary many of its flaws are well documented. By an objective analysis we hope to document not only the impact of Aarogya Setu on user privacy and informational privacy but also any intended benefits it may have offered. This may help a model of technology deployment that is focussed in a rights respecting manner, centred in legality and directed towards outcomes rather than coercion and surveillance.

Looking Back

Even early on, voices were being raised against the blanket imposition of the app. On May 1st, 2020, 45 organisations and more than 100 prominent individuals sent a joint representation to the Prime Minister’s office, arguing against the mandatory use of Aarogya Setu, citing issues of consent, privacy, algorithmic accountability, and the risk of function creep. This led to the usage of Aarogya Setu now being on a ‘best effort basis’. However, the fight did not stop here: several other governments and administrations at the local level mandated the use of Aarogya Setu, such as the Noida Authorities, the Delhi courts, and Delhi wholesale liquor vendors, all of which required contestation.  We also discussed how RWAs may be going well beyond their legally defined functions by mandating the use of Aarogya Setu by residents and visitors including domestic workers, drivers, delivery personnel and other service providers.

Lastly, through our strategic litigation, we were also able to file petitions in the Kerala High Court regarding the mandatory usage of Aarogya Setu to the point of criminal penalties, as well as the app’s third party data sharing protocols.

Issues and Efficacy

In April last year, IFF released a working paper on covid surveillance and privacy in India. A section of the working paper analysed several contact tracing apps, including Aarogya Setu. We detailed several vulnerabilities with respect to privacy, information sharing, and data maximisation in the app, while also questioning its usefulness and accuracy at a more fundamental level. We also noted concerns about how the app could become part of a permanent system of surveillance, especially since there was no defined period for data retention and deletion.

When the Standing Committee on IT was actively considering issues related to ‘citizens data security and privacy’, we wrote to the Committee highlighting the three key concerns of the impact on privacy, the lack of a legislative framework, and the threat of mass surveillance and exclusion. After rising public pressure, the Ministry of Electronics and Information Technology finally released the Aarogya Setu Data Access and Knowledge Sharing Protocol, 2020 on May 11th, 2020. However, we thoroughly analysed the protocol to show that it failed to satisfy the basic threshold of legality. This was based on the several issues:

  • Fundamentally, the Protocol has no statutory backing
  • Violation of the proportionality principle through expansive definitions
  • Implementation concerns and the threat of function creep
  • Lack of meaningful incorporation of the principles of necessity and proportionality
  • Excessive collection, processing, storage, and even sharing of personal data
  • An utterly insufficient sunset clause for the protocol

A general lack of transparency pervades the Aarogya Setu ‘ecosystem’, and this can be seen in several contexts. RTI responses to questions about the app generally failed to receive adequate answers. We also had to send a legal notice to the Central Information Commission regarding their refusal to let RTI activist Saurav Das attend the show cause hearing in his complaint which seeks action against government officials who claim to have no information about the creation of the Aarogya Setu app. The Delhi High Court also issued a notice to the Public Information Officers of several ministries in this matter. To address this lack of transparency and accountability, we released the minutes of the meetings of the 9th Empowered Group on Technology and Data Management (received through an RTI) which provided an inside view into the creation and explanation of Aarogya Setu.

Issues have also been raised about the non open-source nature of the app. Even when the source code was released, it was the code for an earlier version of the app that was made available. Furthermore, the server-side code of the app was not available at all!

Meanwhile, the benefits of Aarogya Setu remain unclear. Firstly, with 11.2 million cases and growing as of March 5th, 2021, the diseases can hardly be said to have been contained, especially since the daily caseload peaked in September - long after the introduction of the app. The Central government itself has apologised for opaque and “irresponsible” behaviour with respect to the app. Its accuracy has also been said to be less than stellar. Indeed, the app has been found unable to pick up details about people on “two sides of a shopping mall or even on two sides of a wall”. Experts have also pointed out the weakness of Bluetooth technology with regards to contact tracing, while cyber security professionals have demonstrated persistent vulnerabilities in the app.  There has been criticism of the privacy paradigm from a technical perspective as well: unlike other contact tracing applications, Arogya Setu uses a static identifier (thus reducing the potential for anonymity), while the application also collects significantly more metadata. These measures create large privacy concerns and leave the app vulnerable to attacks.

Additionally, even those who argue for the app note that direct contact tracing methods have been found to be more effective. Furthermore, while the NITI Aayog may have trumpeted the usefulness of the app by pointing out to its prediction of emerging hotspots, the incidence of false positives (through overestimation of risk across large distances surfaces) and indeed even false negatives (through weakened radio signals) cannot be discounted. The failure to provide risk intervals, a standard practice across all scientific endeavours, makes questions about accuracy even more pressing.

To address concerns about its efficacy, the government said that positivity rate of the application was 27%, more than three times the national average of 8%. While this may look good on paper, it must be noted that such overestimation is not harmless. Such a broad testing interval may disproportionately increase the load on India's creaking testing infrastructure. Moreover, when such significant harms to citizens' privacy is being made, such laxity cannot simply be 'press-conferenced' away.

What does the future hold?

As if this menagerie of serious problems is not enough, the scope of Aarogya Setu has now been further expanded: from its origin as a contact tracing app, Aarogya Setu is now going to be used in the COVID-19 vaccine rollout programme. Vaccination certificates will also be digitally available on the app. Our fears have been confirmed: Aarogya Setu is likely going to become a permanent part of our health data landscape. With its integration with the CoWIN platform, we are going to witness the further consolidation of personal information datasets that can be used for passive surveillance and may lead to large scale exclusion.

This is why we have written to the National Informatics Centre, pointing out that issues with Aarogya Setu have still not been addressed and asking for information on the following issues:

  1. Whether the government plans to phase out Arogya Setu or retain it as a part fo the health data landscape'
  2. Whether the data of inactive users is being deleted under the Protocol;
  3. Whether the data of users for whom the 180 day period has passed been permanently deleted;
  4. Whether the processing of contact, location, and self-assessment data is bundled together or whether they follow separate timelines;
  5. Whether the data of those who have requested the deletion of their demographic data has been deleted;
  6. Whether an audit has been conducted to check whether data taken from the the Arogya Setu database has been deleted;

Important Documents

  1. IFF's letter to the National Informatics Centre regarding the storage of Arogya Setu data (link)