Jaety Edwards

(or John when I have to be official)

I am a computer science researcher with special interest in the fields of Computer Vision and Machine Learning. I received my PhD in 2007 from U.C. Berkeley, where I was advised by Prof. David Forsyth. I now work as Director of Software Development for O.N. Diagnostics, a bio-engineering company designing better tests of bone strength.


My research in graduate school centered on novel approaches for automatically organizing and interacting with large multimedia collections--i.e. collections that contain both text and images.

Searching Handwritten Manuscripts
It's easy for people to teach themselves to read a new script, even if they've never seen it before. Computers, on the other hand, need a lot of hand-holding to achieve the same task (in the form of lots of labelled examples). There are increasing numbers of scanned handwritten manuscripts available online that remain unsearchable because traditional OCR fails and its too time consuming to retrain them. We are investigating models that allow computers to learn new script models without all the help from us.

Collections of Words and Pictures
The web provides access to enormous, previously inaccesible collections whose sheer size makes them difficult to manipulate. Many of these contain images (or video) as well as text. We investigate novel methods of automatically organizing these types of multimedia collections for the purposes of browsing and information extraction. Here, we can utilize tools from both the Natural Language Processing and Computer Vision communities, and we are motivated by the fact that each side tends to reveal information that the other finds very difficult to extract. For example, A writer generally assumes his reader knows a tiger is orange. Thus this fact is almost never given in text, and a computer could never learn it with NLP techniques. However with an associated image, and a technique to link words to image regions, a computer can learn this fact. In the opposite direction, a computer can reliably extract faces from news photos but recognizing faces in uncontrolled environments is still a wide open computer vision problem. With an associated caption, however, we face the much reduced problem of identifying potential names in the caption and then associating each face with this small set of extracted candidates.

Some papers in this area