2024-10-24
This is a must-read. It's hard to overstate just how cheap and easy it is now for hostile actors to corrupt the information environment. Platforms have a role to play countering this, but this is a societal problem. https://www.washingtonpost.com/ ...
Wired
Microsoft: Russia, China, and Iran are stepping up cyber-influence efforts on the US election; China has focused on down-ballot candidates and Congress members
Russia, Iran, and China are targeting the US election with an evolving array of influence operations in the last days of campaign season.
This is a must-read. It's hard to overstate just how cheap and easy it is now for hostile actors to corrupt the information environment. Platforms have a role to play countering this, but this is a societal problem. https://www.washingtonpost.com/ ...
Washington Post
Documents show direct ties between Russia's government and a former Florida deputy sheriff behind a network of fake news sites using AI and targeting US voters
hard to see how this guy doesn't qualify: — https://www.washingtonpost.com/ ... X: Shayan Sardarizadeh / @shayan86 : BBC Verify has extensively reported on an AI-powered influenc...
2024-09-17
This is interesting for many reasons, but particularly so in the wake of the charges related to Russian espionage. Meta would likely not simply rely on accusations made in a charging document to inform this sort of decision (unless related sanctions change the legal calculus). …
NBC News
Meta bans Russian media outlet RT from its apps globally, days after the Biden administration accused RT of acting as an arm of Moscow's spy agencies
U.S. officials on Friday accused RT of carrying out covert information warfare operations around the world as an arm of Russia's spy agencies.
2024-09-13
The deception section of the system card is really fascinating...
Transformer
OpenAI's o1 System Card: “medium” rating for chemical, biological, radiological, nuclear weapon risk, and it sometimes manipulated task data to fake alignment
RE: https://www.threads.net/... X: Max Schwarzer / @max_a_schwarzer : The system card ( https://openai.com/...) nicely showcases o1's best moments — my favorite was when the model ...
The deception section of the system card is really fascinating...
Simon Willison's Weblog
OpenAI's o1 models aren't as simple as the next step up from GPT-4o as they introduce major cost and performance trade-offs in exchange for improved “reasoning”
OpenAI released two major new preview models today: o1-preview and o1-mini (that mini one is also a preview …
2024-09-10
This is a big deal. One of the key nodes of white supremacist accelerationism.
Reuters
US charges leaders of white supremacist gang The Terrorgram Collective, which uses Telegram to celebrate attacks on minorities, with trying to incite a race war
and the state of far-right domestic terrorism — is to operate in a kind of decentralized, leaderless movement. In fact, the structure of so many attacks proves that leaders are cir...
2024-08-29
ISIS had various distribution groups on pretend-encrypted Telegram. Propaganda would start in the inner circle and flow outwards. At that point various folks would distribute the propaganda on other sites, including Facebook. Again, tons of research on this in the terrorism studies community - not secret knowledge. 3/n
Financial Times
Sources: the EU is investigating if Telegram gave inaccurate user numbers to stay under the DSA's 45M threshold; Telegram reported 41M EU users in February 2024
Brussels probing possible failure by messaging platform to provide accurate data under new regulations
My insight was that there was a time gap between the information appearing on Telegram and it showing up on Facebook. Every other site on the internet was downstream of Telegram. So, if we could gather that data on Telegram and process it quickly we didn't need fancy classifiers, we could look directly for the images and text of the ISIS propaganda. Setting up the operations for this was not easy (and I didn't have to do the hardest parts), but we got it done. 4/n
Financial Times
Sources: the EU is investigating if Telegram gave inaccurate user numbers to stay under the DSA's 45M threshold; Telegram reported 41M EU users in February 2024
Brussels probing possible failure by messaging platform to provide accurate data under new regulations
It was no secret that ISIS' primary digital home was Telegram. That was true in 2016, it was true in 2020, and it is still largely true today. The research community that studied ISIS knew this intuitively. It was not secret knowledge. BUT, Silicon Valley didn't really know this. The instinct to counter digital harms was to build classifiers. When I arrived at Meta, I knew we could get at the low-hanging fruit differently, because I knew how ISIS propaganda worked. 2/n
Financial Times
Sources: the EU is investigating if Telegram gave inaccurate user numbers to stay under the DSA's 45M threshold; Telegram reported 41M EU users in February 2024
Brussels probing possible failure by messaging platform to provide accurate data under new regulations
I asked the team to build dashboards focused on time. How long between propaganda being posted on TG and the first upload on FB? How long did it take us to process and hash? It was ultimately really effective, though obviously this system had gaps - unofficial propaganda, weird adversarial edits, etc. This wasn't the end all be all - but it was a key cornerstone that made a big dent in the problem. 6/n
Financial Times
Sources: the EU is investigating if Telegram gave inaccurate user numbers to stay under the DSA's 45M threshold; Telegram reported 41M EU users in February 2024
Brussels probing possible failure by messaging platform to provide accurate data under new regulations
Pavel Durov's detention in France is tied primarily to charges about failing to remove CSAM, but Telegram has also been the central digital home to ISIS...for a decade. It was so central that when I joined Meta (then FB) in 2016, the initial strategy we built to push ISIS off Meta was built, fundamentally, around the centrality of Telegram to ISIS propaganda. And, friends, it worked. Certainly not perfectly, but about as well as could be hoped. Here's the story: 1/n
Bloomberg
France charges Pavel Durov for complicity in the spread of CSAM and other crimes on Telegram; Durov posted a €5M bail and prosecutors banned him from traveling
- Pavel Durov isn't responsible for crimes on app, lawyer says — Durov out of detention on €5 million bail; can't leave France
It's of course also worth noting that Telegram's significant security flaws must have created a dilemma for intelligence agencies. ISIS used this tool, apparently thinking it was secure. It was not. So, do you watch or do you disrupt? ISIS has used other platforms over the years, but Telegram remains attractive to them. I don't know as much about the CSAM activity on Telegram, but this story illustrates why I think of TG in a category of irresponsibility all its own. 8/8
Bloomberg
France charges Pavel Durov for complicity in the spread of CSAM and other crimes on Telegram; Durov posted a €5M bail and prosecutors banned him from traveling
- Pavel Durov isn't responsible for crimes on app, lawyer says — Durov out of detention on €5 million bail; can't leave France
But here's the thing: the process I just described is EXPENSIVE. Only Meta and a few other companies could imagine doing this. The obvious chokepoint in the system was Telegram. If you wanted to disrupt this system, you needed cooperation from Telegram - and with very few exception, Telegram did not cooperate. You can go back and look at the few times there was concerted action. 7/n
Bloomberg
France charges Pavel Durov for complicity in the spread of CSAM and other crimes on Telegram; Durov posted a €5M bail and prosecutors banned him from traveling
- Pavel Durov isn't responsible for crimes on app, lawyer says — Durov out of detention on €5 million bail; can't leave France
There are lots of vendors that track terrorist group activity on Telegram; they could send us the raw data. We didn't trust them to identify ISIS material, so that data was processed very quickly by ops teams, sorted into various buckets and hashed. Eventually, we could do this faster than ISIS was even getting stuff onto FB, so we could literally catch the very first uploads of this propaganda as it was being uploaded. Some would be removed immediately; in other cases sent for review. 5/n
Bloomberg
France charges Pavel Durov for complicity in the spread of CSAM and other crimes on Telegram; Durov posted a €5M bail and prosecutors banned him from traveling
- Pavel Durov isn't responsible for crimes on app, lawyer says — Durov out of detention on €5 million bail; can't leave France
ISIS had various distribution groups on pretend-encrypted Telegram. Propaganda would start in the inner circle and flow outwards. At that point various folks would distribute the propaganda on other sites, including Facebook. Again, tons of research on this in the terrorism studies community - not secret knowledge. 3/n
Bloomberg
France charges Pavel Durov for complicity in the spread of CSAM and other crimes on Telegram; Durov posted a €5M bail and prosecutors banned him from traveling
- Pavel Durov isn't responsible for crimes on app, lawyer says — Durov out of detention on €5 million bail; can't leave France
My insight was that there was a time gap between the information appearing on Telegram and it showing up on Facebook. Every other site on the internet was downstream of Telegram. So, if we could gather that data on Telegram and process it quickly we didn't need fancy classifiers, we could look directly for the images and text of the ISIS propaganda. Setting up the operations for this was not easy (and I didn't have to do the hardest parts), but we got it done. 4/n
Bloomberg
France charges Pavel Durov for complicity in the spread of CSAM and other crimes on Telegram; Durov posted a €5M bail and prosecutors banned him from traveling
- Pavel Durov isn't responsible for crimes on app, lawyer says — Durov out of detention on €5 million bail; can't leave France
Pavel Durov's detention in France is tied primarily to charges about failing to remove CSAM, but Telegram has also been the central digital home to ISIS...for a decade. It was so central that when I joined Meta (then FB) in 2016, the initial strategy we built to push ISIS off Meta was built, fundamentally, around the centrality of Telegram to ISIS propaganda. And, friends, it worked. Certainly not perfectly, but about as well as could be hoped. Here's the story: 1/n
Financial Times
Sources: the EU is investigating if Telegram gave inaccurate user numbers to stay under the DSA's 45M threshold; Telegram reported 41M EU users in February 2024
Brussels probing possible failure by messaging platform to provide accurate data under new regulations
It's of course also worth noting that Telegram's significant security flaws must have created a dilemma for intelligence agencies. ISIS used this tool, apparently thinking it was secure. It was not. So, do you watch or do you disrupt? ISIS has used other platforms over the years, but Telegram remains attractive to them. I don't know as much about the CSAM activity on Telegram, but this story illustrates why I think of TG in a category of irresponsibility all its own. 8/8
Financial Times
Sources: the EU is investigating if Telegram gave inaccurate user numbers to stay under the DSA's 45M threshold; Telegram reported 41M EU users in February 2024
Brussels probing possible failure by messaging platform to provide accurate data under new regulations
I asked the team to build dashboards focused on time. How long between propaganda being posted on TG and the first upload on FB? How long did it take us to process and hash? It was ultimately really effective, though obviously this system had gaps - unofficial propaganda, weird adversarial edits, etc. This wasn't the end all be all - but it was a key cornerstone that made a big dent in the problem. 6/n
Bloomberg
France charges Pavel Durov for complicity in the spread of CSAM and other crimes on Telegram; Durov posted a €5M bail and prosecutors banned him from traveling
- Pavel Durov isn't responsible for crimes on app, lawyer says — Durov out of detention on €5 million bail; can't leave France
It was no secret that ISIS' primary digital home was Telegram. That was true in 2016, it was true in 2020, and it is still largely true today. The research community that studied ISIS knew this intuitively. It was not secret knowledge. BUT, Silicon Valley didn't really know this. The instinct to counter digital harms was to build classifiers. When I arrived at Meta, I knew we could get at the low-hanging fruit differently, because I knew how ISIS propaganda worked. 2/n
Bloomberg
France charges Pavel Durov for complicity in the spread of CSAM and other crimes on Telegram; Durov posted a €5M bail and prosecutors banned him from traveling
- Pavel Durov isn't responsible for crimes on app, lawyer says — Durov out of detention on €5 million bail; can't leave France