Hewlett Packard announced on June 1st it is collaborating with Tsinghua University in Beijing, China, to develop multimedia technologies, including automated photo sorting, facial expression recognition and video and music analysis.
The collaboration brings together two of the world's leading research institutions in multimedia management. HP Labs helped create the company's three basic print technologies - inkjet, LaserJet and liquid electrophotography - and contributed to its entry into digital photography. HP is the acknowledged market leader in home and office printing.
Tsinghua University has leading intellectual property in multimedia research, including analysis, process, search and management.
The joint lab research would rely mainly on image recognition - instead of traditional text identification - for visual applications. The music research would be based on sound.
"With the explosion of both amateur and professional multimedia, we need increasingly sophisticated tools to identify, sort and process the millions of images and sounds that have been collected,"Â said Patrick Scaglia, chief technology officer, Imaging and Printing Group, HP. "We have to rely on multimedia tools to analyze multimedia data, so we're delighted to partner with Tsinghua University, which has world-class skills in this area."Â
"We are looking forward to partnering with HP and particularly in collaborating with HP Labs, which is exploring very advanced multimedia management technologies that we believe will have a major impact in the future,"Â said Kang KeJun, vice president, Tsinghua University, who signed the agreement in a ceremony today at the university. Shane Robison, executive vice president and chief strategy and technology officer, signed on behalf of HP.
The initial research areas include:
Automatic classification of digital photos: Based just on visual analysis, photos in a collection can be sorted into categories such as portraits, group photos, babies, sports, etc. This capability could prove extremely helpful to people or organizations that store thousands of photos - either online or on a PC - without having to label every picture.
Improved face identification: "Show me all the photos of my grandmother."Â A user could enter a query to find all the photos of a certain individual and the system would retrieve them - even if the individual was in photos with other people or his or her face was partially covered. The technology is also being extended to include video.
Video-based audience analysis: Owners of digital signage could use vision technologies to identify how many people stopped by their display, how long they stayed and what their facial expressions revealed --- pleased, upset, surprised, bored, etc.
Video digital warehouse: Videos could by analyzed, sorted, stored and retrieved based on graphic features such as frame, shots and scenes, as well as by content. Researchers will explore the use of very large parallel database technology as an enabler for multimedia data warehouses.
Video search and recommendation for Internet-based video communities: With the proliferation of videos posted online, an algorithm is being developed that would identify similar subject matter to recommend to viewers within a given user community. For example, someone who watched a certain sports video might be offered similar clips.
Music analysis and retrieval: A user could tell the system, "find me more music like this,"Â and play a sample. The system would then provide recommendations, based on rhythm, melodies, vocals, instruments or other audio elements.
The agreement is funded by HP's Imaging and Printing Group, Personal Systems Group and HP Labs. During the collaboration, Tsinghua University researchers are expected to spend time at HP Labs China in Beijing as well as with members of the Imaging Technology Department in HP Labs in Palo Alto. HP researchers will also go to China and work at the university.
HP has worked with the university on a variety of projects over the past 10 years, including the ChinaGrid, which is intended to extend information technology resources and services to thousands of researchers and the more than 20 million students in the country's university system. It has also worked with the university on hydroinformatic management systems for controlling China's vast water resources.
In recent years, HP has set up several research and development organizations in China, including HP Labs China, the HP Global Delivery China Center, China Design Center and the Printing and Imaging Research and Development Institute. The organizations have been involved in both products designed for China and software and solutions for the global market. -Hewlett Packard
Pictures for this story
