Email Updates RSS
Search
Dr. Hai-Jew discusses her research on coding imagery and multimedia.

Do Multimedia and Imagery Mean More than Meets the Eye?

By Taylor Chernisky on Aug 30, 2017
coding word cloud Every second, your brain processes 400 billion bits of information. This information can be words that you see or hear, graphics, or images. In order to process this information, your brain must be able to quickly analyze the information and determine its relevance to you, and then either code it into your memory or discard it. Considering that we are only aware of 2,000 of these bits of information per second, it is safe to say that our brains do more discarding than coding.

Dr. Shalin Hai-Jew, editor of multiple IGI Global books, has recently authored a new publication, Techniques for Coding Imagery and Multimedia: Emerging Research and Opportunities, that looks at how images and multimedia information can be coded in order to be properly understood and categorized. She recently talked with IGI Global about the scope of this research, as well as some future opportunities with this research and why it is important to study.
Q: What does it mean to “code” imagery and multimedia?

Coding here, generally, refers to the extraction of meaning from the imagery and multimedia by analyzing the works and finding patterns. Coding work may also involve the categorizing and labeling of the respective works. In the same way that people encode meanings in writing, they also encode meanings in their imagery and in multimedia objects. While coding itself is already very difficult and complex in the analysis of texts, the high dimensionality of imagery and multimodal and mixed forms of media adds that much more complexity and difficulty. Those who use an emergent, bottom-up coding approach with such digital contents will have to contend with a variety of messages and dimensions, often with very few computational supports for their coding work. Some may argue that coding is also about co-constructing meaning from the digital objects as “prompts,” without the implication of inherent built-in meanings to the respective works. And of course, it is important to document and define these efforts for the Methods sections of papers and to support others’ research works, both contemporaneously and into the future.

It strikes me that there are lots of right ways to do this—to code image and multimedia “data,” if you will. I think that it is okay to explore and experiment, to go with some initial naïve approaches, to be banal, even, in coding, so as to lay the groundwork for more nuanced and sensitive work later on. To generalize, though, if done manually, this is painstaking and iterative work. It really helps to have a system and a plan that is informed by both a priori assumptions as well as the image and multimedia sets.

Q: How do you source imagery for research? Likewise, how do you source multimedia for research?

There were a number of sources for the digital research materials in this book, including the Web, Wikipedia, article collections from a media organization, Instagram, and others. It’s pretty remarkable how many web browser add-ons, as well as software add-ons, are out there to enable the collection of data. If you tend towards “greedy” when it comes to data collection, as I tend to be, the technologies enable a pretty mass grab of digital stuff.

Much of what is out in the world today is multimedia. There are very few objects that are purely one thing or another (no pure texts without imagery, no slideshows without some audio overlay, no video without some HTML interactivity, etc.). There is a lot going on in combinatorial ways, with resulting combinatorial complexities. The engaging with multimedia opens up new ways of thinking, with integrated modalities to explore, express, and analyze.

Q: What is quality “coding” in relation to imagery? What is quality “coding” in relation to multimedia?

“Quality” coding depends on the actual research context and what the researchers are looking for and trying to study. There are generalizable points about quality. I think “quality” has something to do with coding accuracy and with coding methods (and how accepted these methods are in particular domains). There is something about coding efficiency that has to be achieved…and maybe something about reproducibility (to a degree). Ultimately, learned insights are important as well. You want to code towards relevance, not just pattern identification.

Coding Word Tree Visualization

Q: Could you tell us about Techniques for Coding Imagery and Multimedia: Emerging Research and Opportunities?

Sure, the book is organized in three sections: (1) Coding Imagery for Sensemaking, (2) Exploring Social Phenomena, and (3) Image and Multimedia Coding in Academia. It features seven chapters and a preface and an epilogue. Topically, there are explorations of a #selfie #humor image-set from Instagram, an exploration of snacks around the world, technology manifestos, American citizenship renunciation, and academic assessment in evaluating student-created imagery and multimedia. The topics range broadly because I needed to seed the explorations in order to collect data and to apply the coding techniques, but at heart, this work is about applied coding of imagery and multimedia.

Q: What are some necessary skills for coding imagery and multimedia? Are these skills easy to develop?

On the one hand, I want to argue that humans have an inherent capability for visual analysis, and they do. Yet, I also know that researchers are made, not born…and I know a lot of training goes into the work of research, coding, and analysis. So I’ll put myself somewhere between those two assertions, closer to the training end. That said, the technologies described in this work are all publicly available tools. The commercial software has high performance and functionality. Some of the technology is free and open-source. What is cutting-edge now may not really even manifest in some locales around the world until years and years later. What is faddish now may recur in later waves, in different forms, and in different geographies, and among different peoples. There are valuable insights from local contexts that are valuable even if the work is not bleeding edge.


Q: How did you get interested in this area of study?

In my work in instructional design, I am in online spaces a lot, and I end up doing a lot of different things. As is typical with book proposals, I saw a gap and thought it might be fun to see what was out there in terms of coding complex digital objects, like imagery and multimedia. I thought, “Wow, I’ll bet a lot of people are coding imagery and multimedia as part of their research, and I’ll bet they’ll be willing to share some techniques!” This was back in 2015. It turned out that while I think the first part of my hunch was right, the latter part was not, thus, an edited book turned into an authored one. I would have preferred having a wide stable of authors for a range of insights, but I think there is still a lot of diversity in this work.

I’m all about going for the best that you can achieve at a time, with what’s available, and then not regretting what could have been. In this time period, I also had a partial retinal detachment. During this book’s gestation, I spent part of that time recovering and learning to appreciate just how critical human vision can be for some types of coding and analysis.

I want to be ambitious in question-asking and not to stop myself just because a work seems hard or initially impossible.


Q: Why do you believe that this research is important in today’s world?

Having some efficient ways to code imagery and multimedia—with a combination of manual and computational methods—is important since so many researchers are exploring relevant research questions with high-dimensional data. While there’s a lot of value in studying small data sets and even singular examples, there are different types of insights attainable from large structured and semi-structured datasets. Researchers learn from each other’s methods and approaches all the time, and this is one area in which researchers’ methods can be high-impact and valuable to others. We’re in an age when there is so much raw digital data available out there, much of it consumed locally only, that we need more effective methods to harness this. In the same way that there is a “great unread” in terms of books, there is a “great unread” of social shared digital data. There is a lot of niche content—stuff in the “long tail”—that is wholly unexplored. The little that automation enables exploration of is seen by those in corporations and government but not so much common researchers because of prohibitive costs…and the coarse scale of the analytics (often focused around public personalities and brands and politics).

Q: Is this research relevant to any trending topics in today’s news?

Actually, quite so! With so much talk about #fakenews, those in mass media and in information-based organizations are working hard to separate the wheat from the chaff in terms of public opinions. So much in a modern democracy rides on public awareness and public opinion. Public opinion polls are still very much with us and relevant, but so too are “human sensor networks” in terms of postings on social media platforms. While three are automated ways to extract meaning from texts, imagery, and video, at scale, those approaches are not low-cost and not particularly nuanced. For smaller sets of semi-structured data (like imagery and multimedia files), human-led analytics can be much more effective and complex and culture-informed. This work shows ways to get closer to #realnews and #realthoughts using informed, interpretive, and systematic analytical approaches to imagery and multimedia contents.

Q: What are some of the research opportunities in this area?

We need to find efficient ways to find meaning from digital contents from social media platforms. Based on coding imagery and multimedia, we need to find ways to answer particular research questions across all domains. We need ways to provide fair and efficient assessments of multimodal student work. We also need to harness various technologies to enable efficient coding of images and multimedia, without the fairly huge amounts of noise based on what is available today. (While IBM Watson’s API does enable access to some impressive machine coding, not all questions can be efficiently answered through that tool.)

I think this book can be a positive conversation starter, to see how people code imagery and multimedia where they are. My “local maxima” is not the “global maxima,” and there is a lot of room for us to learn from each other. Together, we can contribute to defining this space.
IGI Global would like to thank Dr. Hai-Jew for taking the time to discuss her recent publication with us. Please take a moment to review Dr. Hai-Jew’s publications below:

Newsroom Contact
Caroline Campbell
Promotions Assistant
ccampbell@igi-global.com
(717) 533-8845, ext. 144
www.igi-global.com/
Browse for more posts in:
Media and Communications

No comments Comments

Log in or sign up to comment.
Be the first to comment!

More from IGI Global

Jul 1, 2024 Ethicality Issues of Withdrawing Articles After Acceptance
There is a growing issue in the research community of authors submitting their articles to gold OA journals (and even hybrid OA/subscription-based journals), and then withdrawing their article after it has been accepted to the journal, effectively utilizing all of the resources of that journal, including the peer review, only to then refuse to publish at the end of the process.
Erin WatsonRead More
NewsletterAuthor NewsOpen Access
Jun 27, 2024 The Truth Behind California’s Plastic Pollution Reduction Plan
The sheer amount of plastic waste has become overwhelming and newly proposed laws in California will only make matters worse.
IGI GlobalRead More
Business and ManagementEnvironmental, Agricultural, and Physical SciencesSustainable DevelopmentBooks & E-Books
Jun 21, 2024 IGI Global Announces the 2023 Journal Reviewer Award Winners
IGI Global congratulates the winners of this year's Journal Reviewer Award
IGI GlobalRead More
JournalsAwards & RecognitionOpen Access
Jun 20, 2024 How Do You Get Published in High Impact Journals?
IGI Global and eContent Pro held an enlightening webinar that offered pointers for researchers, academicians, and professionals looking to increase their chances of having their manuscript accepted into high impact scholarly journals.
IGI GlobalRead More
EducationBusiness and ManagementComputer Science and Information TechnologyJournalsWebinar
Jun 20, 2024 The Right to Read
Oklahoma City school district wins case against the state Board of Education.
IGI GlobalRead More
EducationSocial Sciences and HumanitiesBooks & E-Books
Jun 13, 2024 Why Are Researchers Merging AI with Satellite Image Surveillance?
Australian scientists create new bushfire detection method through the use of artificial intelligence and satellites.
IGI GlobalRead More
Environmental, Agricultural, and Physical SciencesScience & EngineeringArtificial IntelligenceBooks & E-BooksResearch Trends
Jun 12, 2024 Q&A with Dr. Rana Zeine: Medical Cannabis Research for Cancer, MS, Epilepsy & More
Dr. Zeine has spoken about her research at several events. Most recently, she was a Guest Speaker at the SALIHIN Corporate Centre in Kuala Lumpur, Malaysia, providing insights and background related to the book. See below, where she answers some of our questions about the conception of this research book.
IGI GlobalRead More
Medical, Healthcare, and Life SciencesBooks & E-Books
Jun 6, 2024 Treaty Signed After a Decade of Fighting
World Intellectual Property Organization (WIPO) signs first treaty in a decade, discusses link between genetic resources and Indigenous knowledge.
IGI GlobalRead More
Medical, Healthcare, and Life SciencesBooks & E-BooksResearch Trends
Jun 3, 2024 How Do You Get Published in High Impact Journals?
IGI Global and eContent Pro are holding a free webinar on Wednesday, June 12th at 10:30 AM EST via Microsoft Teams.
IGI GlobalRead More
JournalsWebinar
May 31, 2024 IGI Global Titles Included in Doody's Core Titles 2024
Recently, Doody's Core Titles, an essential collection development tool for health science librarians, included 5 IGI Global reference books in their book database.
IGI GlobalRead More
Medical, Healthcare, and Life SciencesBooks & E-BooksAwards & RecognitionResources for Librarians
First Previous 1 2 3 4 5 6 7 8 9 10  ... Next Last