Visualizing an image network without rendering files: A method to combine user hashtags with computer vision labels
DOI:
https://doi.org/10.7203/drdcd.v1i8.237Resum
This article presents a method for visualizing networks of geolocated images without rendering the image files on the network. The path I followed to develop this method is the result of an intensive "data sprint" which took place during the University of Amsterdam Digital Methods Initiative Summer School 2021. During the data sprint, I developed a methodological framework to generate a network of Twitter geolocated images combining the hashtags twitted with the images and the Google Cloud Vision API best single expression to describe each image (BestGueesLabel). Considering the limitations of working with a massive amount of image data and the computational memory required to generate network visualizations, the possibility of using description tags to create image networks is promising. The images analyzed during this study were extracted from Twitter filtering for the #deepfakes and #deepfake and tagged with country code location. Thus, the hashtags included in the tweets by Twitter users provide the context and the user description of the image. This information was combined in a bipartite network with a computer vision entity, the computer vision description of the image, to generate a networked description of the whole image set. I point that this method can be considered in exploratory research when working with large sets of images.
Referències
Bimber, Bruce, and Homero Gil de Zúñiga. 2020. “The Unedited Public Sphere.” New Media & Society 22(4):700–715. doi: 10.1177/1461444819893980.
Borgos, J. B. (2021) country-flags Available at: https://github.com/hampusborgos/country-flags (Accessed: 08 Jul. 2021).
Brunner, Thomas, Frederik Diehl, Michael Truong Le, and Alois Knoll. 2019. “Guessing Smart: Biased Sampling for Efficient Black-Box Adversarial Attacks.” Pp. 4957–65 in 2019 IEEE/CVF International Conference on Computer Vision (ICCV). Seoul, Korea (South): IEEE. doi: 10.1109/ICCV.2019.00506.
Chao, J. (n.d) memespector-gui. Available at: https://github.com/jason-chao/memespector-gui (Accessed: 08 Jul. 2021).
Chao, J., Omena, J.J. (2021) Networks of Image Description. Available at: https://docs.google.com/document/d/1ccBH691tW-6jVTrjQtMjd2aDTalZolIbRBtMqIGISIg/ (Accessed: 05 Dec. 2021).
Chesney, R., Citron, D.K. (2018), Deep Fakes: A Looming Challenge for Privacy, Democracy, and National Security, SSRN Scholarly Paper No. ID 3213954, Social Science Research Network. Rochester, NY. doi: /10.2139/ssrn.3213954.
chezsick. 2017. “Gephi 0.9.1 Can Never Increase the Memory · Issue #1705 · Gephi/Gephi.” GitHub. Retrieved April 8, 2022 (https://github.com/gephi/gephi/issues/1705).
Cole, S. (2017). AI-Assisted Fake Porn Is Here and We’re All Fucked. Vice. Available at: https://www.vice.com/en/article/gydydm/gal-gadot-fake-ai-porn (Accesse:d 05 Dec. 2019).
Dasilva, Jesús Pérez, Koldobika Meso Ayerdi, and Terese Mendiguren Galdospin. 2021. “Deepfakes on Twitter: Which Actors Control Their Spread?” Media and Communication 9(1):301–12. doi: 10.17645/mac.v9i1.3433.
Geboers, Marloes Annette, and Chad Thomas Van De Wiele. 2020. “Machine Vision and Social Media Images: Why Hashtags Matter.” Social Media + Society 6(2):205630512092848. doi: 10.1177/2056305120928485.
Gibbs, M., Meese, J., Arnold, M., Nansen, B. and Carter, M. (2015), “#Funeral and Instagram: death, social media, and platform vernacular”, Information, Communication & Society, Vol. 18 No. 3, pp. 255–268. doi: 10.1080/1369118X.2014.987152.
ghost. 2016. “Gephi Memory Issue · Issue #1609 · Gephi/Gephi.” GitHub. Retrieved April 8, 2022 (https://github.com/gephi/gephi/issues/1609).
Google. (2020) Google.Apis.Vision.v1.Data.WebDetection Class Reference. Available at: https://developers.google.com/resources/api-libraries/documentation/vision/v1/csharp/latest/classGoogle_1_1Apis_1_1Vision_1_1v1_1_1Data_1_1WebDetection.html#afddf64d68931805b23d9c863dd47b9c7 (Accessed: 05 Dec. 2021).
Graham M., Hale S. A. and Gaffney D. (2014) Where in the World Are You? Geolocation and Language Identification in Twitter, The Professional Geographer, 66:4, 568-578, doi: 10.1080/00330124.2014.907699.
Ilyas, Andrew, Logan Engstrom, Anish Athalye, and Jessy Lin. 2018. “Black-Box Adversarial Attacks with Limited Queries and Information.” Pp. 2137–46 in Proceedings of the 35th International Conference on Machine Learning. PMLR. https://proceedings.mlr.press/v80/ilyas18a.html.
Jung, K. (2021) PicArrange -- Visually Sort, Search, and Explore Private Images on a Mac. arXiv:2111.13363
KallyopeBio. 2018. “Gephi UI Nearly Unresponsive with Large Network (100% CPU Utilization on One Core) · Issue #1947 · Gephi/Gephi.” GitHub. Retrieved April 8, 2022 (https://github.com/gephi/gephi/issues/1947).
Kearney, M.W. (2017) rtweet. Available at: https://github.com/mkearney/rtweet.download (Accessed: 23 Mar. 2018)
Kietzmann, Jan, Linda W. Lee, Ian P. McCarthy, and Tim C. Kietzmann. 2020. “Deepfakes: Trick or Treat?” Business Horizons 63(2):135–46. doi: 10.1016/j.bushor.2019.11.006.
Kim, A.E., Hansen, H.M., Murphy, J., Richards, A.K., Duke, J. and Allen, J.A. (2013), “Methodological Considerations in Analyzing Twitter Data”, JNCI Monographs, Vol. 2013 No. 47, pp. 140–146. doi: 10.1093/jncimonographs/lgt026.
Mack, David. 2018. “This PSA About Fake News From Barack Obama Is Not What It Appears.” BuzzFeed News. Retrieved April 8, 2022 (https://www.buzzfeednews.com/article/davidmack/obama-fake-news-jordan-peele-psa-video-buzzfeed).
Marres, Noortje. 2020. “For a Situational Analytics: An Interpretative Methodology for the Study of Situations in Computational Settings.” Big Data & Society 7(2):2053951720949571. doi: 10.1177/2053951720949571.
Mintz, André, and Tarcizio Silva. 2019. Interrogating Vision APIs. Lisboa: Universidade Nova de Lisboa | NOVA FCSH | iNOVA Media Lab.
Niederer, Sabine, and Gabriele Colombo. 2019. “Visual Methodologies for Networked Images: Designing Visualizations for Collaborative Research, Cross-Platform Analysis, and Public Participation.” Diseña (14):40–67. doi: 10.7764/disena.14.40-67.
Omena, J.J. (2019) Introdução: O que são Métodos Digitais? In: Métodos Digitais: Teoria-Prática-Crítica, Ed. Janna Joceli Omena. Lisboa: ICNOVA.
Pearce, W, Colombo, G & De Gaetano, C 2019, 'Using computer vision to see Google’s visual vernacular of climate change (2008-19)'. https://research.hva.nl/en/publications/using-computer-vision-to-see-googles-visual-vernacular-of-climate.
Rogers, R. (2009) ROGERS, R. The end of the virtual: Digital Methods. Inaugural lecture, University of Amsterdam. doi: 10.5117/9789056295936.
Rogers, R. (2021) ‘Visual media analysis for Instagram and other online platforms’, Big Data & Society. doi: 10.1177/20539517211022370.
Silva, Tarcízio, André Mintz, Janna Joceli Omena, Beatrice Gobbo, Taís Oliveira, Helen Tatiana Takamitsu, Elena Pilipets, and Hamdan Azhar. 2020. “investigando mediações algorítmicas a partir de estudo de bancos de imagens.” Logos 27(52):30. doi: https://doi.org/10.12957/logos.2020.51523.
Stevens, G., Rohde, M., Korn, M. and Wulf, V. (2018), “Grounded Design: A Research Paradigm in Practice-Based Computing”, Socio-Informatics. Oxford University Press, Oxford. doi: 10.1093/oso/9780198733249.003.0002.
Tucci, G., Roth, M., Saxler, F. (2021) Geographical mapping: Twitter hashtags and Google Vision web entities. In: Mapping deepfakes with digital methods and visual analytics, Digital Methods Initiative. Available at: https://wiki.digitalmethods.net/Dmi/WinterSchool2021Deepfakes#A_5.3.3._Geographical_mapping:_Twitter_hashtags_and_Google_Vision_web_entities (Accessed: 05 Dec. 2021).
Vaccari, Cristian, and Andrew Chadwick. 2020. “Deepfakes and Disinformation: Exploring the Impact of Synthetic Political Video on Deception, Uncertainty, and Trust in News.” Social Media + Society 6(1):205630512090340. doi: 10.1177/2056305120903408.
Westerlund, M. (2019). The Emergence of Deepfake Technology: A Review. Technology Innovation Management Review, 9(11), pp. 40–53. doi: 10.22215/timreview/1282.
Xue, Cris. (2012) gephi-plugin-image-preview. Available at: https://github.com/chrisxue815/gephi-plugin-image-preview (Accessed: 08 Jul. 2021).
