Nobody — not even those who have access to the code and data — can tell what piece of data came together with what other piece of data to result in the finding the program made. This further undermines the notion of informed consent, as we do not know which data results in what privacy consequences. This creates an incentive for companies to collect and store as much data as possible, and to bury the privacy ramifications, either in legalese or by playing dumb and being vague. And there should also be significant penalties for data breaches, especially ones that result from underinvestment in secure data practices, as many now do. Companies often argue that privacy is what we sacrifice for the supercomputers in our pockets and their highly personalized services.

Source: New York Times January 30, 2018 10:46 UTC