As Upguard writes, this raises two important questions: why is the DoD spying on everyone (including US citizens at home and abroad, as well as active-duty service members), and why were they so careless with all the data they amassed through that spying?
The archive appears to originate with the amazingly named, defunct government contractor "VendorX," who appear to have been grossly negligent in the execution of their duties, an incompetence that the Pentagon never seemed to notice.
Massive in scale, it is difficult to state exactly how or why these particular posts were collected over the course of almost a decade. Given the enormous size of these data stores, a cursory search reveals a number of foreign-sourced posts that either appear entirely benign, with no apparent ties to areas of concern for US intelligence agencies, or ones that originate from American citizens, including a vast quantity of Facebook and Twitter posts, some stating political opinions. Among the details collected are the web addresses of targeted posts, as well as other background details on the authors which provide further confirmation of their origins from American citizens.
What is more clear is the significance of these data repositories’ contents.The collection of public internet posts in massive repositories by the Defense Department for unclear reasons is one matter; the lack of care taken to secure them is another. The CENTCOM and PACOM CSTAR cyber risk scores of 542 and 409 provide some indication of gaps in the armor of two major military organizations’ digital defenses. The possible misuse or exploitation of this data, perhaps against internet users in foreign countries wracked by civil violence, is a troubling possibility, as is the presence of US citizens’ internet content in buckets associated with US military intelligence operations. The Posse Comitatus Act restricts the military from “ being used as a tool for law enforcement, except in situations of explicit national emergency based on express authorization from Congress,” but as seen in recent years, this separation has been eroded.
Despite all of this, the same issues of cyber risk driving insecurity across the landscape are present here, too. A simple permission settings change would have meant the difference between these data repositories being revealed to the wider internet, or remaining secured. If critical information of a highly sensitive nature cannot be secured by the government - or by third-party vendors entrusted with the information - the consequences will affect not only whatever government organizations and contractors that are responsible, but anybody whose information or internet posts were targeted through this program, potentially resulting in unfair bias or unwarranted actions against the post creator.
Dark Cloud: Inside The Pentagon's Leaked Internet Surveillance Archive[Dan O'Sullivan/Upguard]