![\"predictim](\"https:\/\/articles-img.sftcdn.net\/sft\/articles\/auto-mapping-folder\/sites\/3\/2018\/12\/ai-for-babysitters-1024x576.jpg\")
<\/p>\n<\/p>\n
So, a few weeks back, the Washington Post ran a story<\/span><\/a> that had a disturbing tinge running through it. The article centered on a software called Predictim<\/a>. It’s a service that scans potential babysitters\u2019 Twitter feeds, Facebook pages, and Instagram stories to provide concerned parents automated risk ratings for things like drug use and bullying.<\/strong><\/p>\n Less objective measures like a bad attitude or disrespectfulness were also included in the report \u2014 which, really?<\/p>\n It\u2019s certainly understandable that parents want to check out the person watching their child. But this technology presents some new issues that go beyond general workplace spying.<\/a><\/p>\n Here\u2019s a little background on Predictim, and a look at what\u2019s looming over the horizon when it comes to new ways to invade peoples\u2019 space.<\/p>\n Predictim scans the digital footprint of a prospective babysitter to determine their risk to parents. The app was created by Sal Parsa and Joel Simonoff who set out to develop an AI solution that can generate personality assessments based on the candidate\u2019s digital footprint.<\/p>\n The app used natural language processing (NLP) to sort through social media platforms, scanning photos, tweets, and captions for risk factors \u2014 Parsa and Simonoff say that traditional background checks or interviews don\u2019t paint the full picture of who a person might be.<\/p>\n With Predictim, the algorithm considers billions of data points and can scan them within a matter of minutes \u2014 spitting out a report with predicted traits, behavior, and a brief summary of their digital history.<\/strong><\/p>\n Reports cost $24.99 a pop and come with a risk assessment score \u2014 green being not risky to red, very risky. A small price to pay for identifying potential bullies and abusers who might otherwise slip through the cracks.<\/p>\n Critics of the tool say that technologies like the solution that Predictim has presented present some clear dangers, especially given that these tools are being used to make decisions about someone\u2019s ability to find work.<\/p>\n The first problem that comes to mind, of course, is bias<\/strong>. Often AI tools don\u2019t show bias until they\u2019ve been deployed and someone starts to pick up on this unfortunate pattern.<\/p>\n Gizmodo\u2019s Brian Merchant decided to experiment with Predictim<\/span><\/a> and his post-mortem highlights some major problems.<\/strong> Merchant says he tested the app using his actual babysitter, an African American woman, and a white male friend. The friend got a better rating than the babysitter, despite the fact that he shares a lot of vulgar content on Twitter. Merchant’s babysitter was flagged for being disrespectful.<\/p>\n Joel Simonoff, the CTO, says that they don\u2019t look at things like race or ethnicity, but we\u2019ve seen other tools develop biases, too.<\/strong> Amazon\u2019s recruitment AI didn\u2019t especially like women and software used by many of the nation\u2019s prisons routinely rates black inmates as more likely to commit a crime in the future.<\/p>\n Another issue is this tool turns up a lot of personal data.<\/strong> Predictim pulls up addresses, phone numbers, email addresses, and names of relatives \u2014 to, you know, ensure that the person running the scan knows, just for sure, that they\u2019ve got the right person.<\/p>\n Finally, these recruitment apps are classist<\/strong>. Increasingly, we\u2019re developing tools that disproportionally subject low wage workers to robot-based scrutiny. While Predictim does need to obtain the babysitter\u2019s consent before processing the report, if the sitter opts out\u2014surprise, they\u2019re not going to get the job.<\/p>\n Other tools like HireVue<\/a> and Fama<\/a> work with companies\u2014reporting back on things like attitude and demeanor.<\/p>\n Fama scans social media channels for toxic behavior and alerts the bosses, while HireVue analyzes tone and demeanor to predict on the job performance \u2014 candidates are encouraged to smile at the bot for best results.<\/p>\n Your future may depend on your social score<\/p>\r\n Read now \u25ba<\/a>\r\n <\/div>\r\n <\/div>\r\n <\/a>\r\n <\/div>\r\n<\/div>\n Facebook pulled most of Predictim\u2019s access to users, as the company was found to be in violation of the platform\u2019s privacy policy on how personal data can be used. Predictim is still scraping public data for their reports, but Facebook may end up pulling them completely.<\/p>\n Twitter says, when they became aware of the platform, they blocked it as well. A spokesperson for the company said that Twitter prohibits the use of their data and APIs for surveillance purposes<\/a>.<\/p>\n Still, Predictim isn\u2019t giving up. The founders are currently in talks with shared economy companies to provide vetting services for rideshare services or accommodation hosting. We\u2019re sure there are some big names involved.<\/p>\n While it\u2019s heartening that the social media giants have shut this tool out \u2014 there are so many of these vetting platforms that keep cropping up. And, you\u2019re looking at things like a bad attitude or evaluating personalities with algorithms, there\u2019s going to be a problem in the power dynamic between employers and those vying for opportunities from the babysitting gig down the street to gig economy sites and the traditional 9-5.<\/p>\n Black Mirror comes to life with China\u2019s \u2018social credit\u2019 scoring system<\/p>\r\n Read Now \u25ba<\/a>\r\n <\/div>\r\n <\/div>\r\n <\/a>\r\n <\/div>\r\n<\/div>\n It looks like the media scrutiny offered more blowback than Predictim, uh, predicted. Visit the site now and you’ll see a statement at the top:<\/p>\n “We have been overwhelmed by the interest, press coverage, and input regarding our project.\u00a0To be honest, this attention came earlier than we expected, and certainly before we had fully launched our contemplated services.\u00a0We received some very helpful feedback on ways we could make Predictim even better. Clearly, people are hungry for better ways to make decisions in marketplaces where character, reputation, and trustworthiness are important.\u00a0As a result, we have decided to pause our full launch and put our heads down to focus on evaluating how we offer our service and making changes to address some of the suggestions we received.\u00a0While we are not offering any services at this time, please stay tuned and check back often for updates: we will be back!”<\/em><\/p>\n Ultimately, it doesn’t seem like Predictim and similar apps are going away anytime soon. Even if Facebook and Twitter keep blocking the platform from scraping private data, there’s always the chance the companies reach a compromise if there’s enough money on the table.<\/p>\n The fact that enough people are willing to use this service to spy on the neighborhood teen is a bit troubling, and it’s a sure sign that we need to start fighting for our privacy before it’s too late.<\/strong><\/p>\nHow does Predictim work? What\u2019s the deal?<\/h3>\n
![\"predictim\"](\"https:\/\/articles-img.sftcdn.net\/sft\/articles\/auto-mapping-folder\/sites\/3\/2018\/12\/riskyrebecca.jpg\")
<\/p>\nWhat are some of the issues?<\/h3>\n
![\"predictim\"](\"https:\/\/articles-img.sftcdn.net\/sft\/articles\/auto-mapping-folder\/sites\/3\/2018\/12\/predictim2.jpg\")
<\/p>\n![\"predictim\"](\"https:\/\/articles-img.sftcdn.net\/sft\/articles\/auto-mapping-folder\/sites\/3\/2018\/12\/predictim.jpg\")
<\/p>\n\r\n \r\n \r\n \r\n ![](\"https:\/\/articles-img.sftcdn.net\/sft\/articles\/auto-mapping-folder\/sites\/3\/2018\/12\/social-score-192.jpg\")
\r\n <\/div>\r\n \r\n Facebook and Instagram blocked the tool<\/h3>\n
\r\n \r\n \r\n \r\n ![](\"https:\/\/articles-img.sftcdn.net\/sft\/articles\/auto-mapping-folder\/sites\/3\/2019\/01\/black-mirror-social-score-192.jpg\")
\r\n <\/div>\r\n \r\n What’s next?<\/h3>\n