Structural Informatics Group, Department of Biological Structure
Division of Biomedical and Health Informatics,
Department of Medical Education
University of Washington School of Medicine
lober@u.washington.edu
supported by NLM fellowship F38 LM00086
So, what are annotated images, and why would anybody want to serve them? Please Read On...
PAIS is the Personal Annotated Image Server. PAIS, I hope, offers an easy way to add comments to your images, and publish those commented images over the web. It is aimed at medical educators, but the principles are generally applicable to anyone with images.
More precisely, PAIS is a simple system for creating a series of annotated images, in a linked image collection, and making them viewable over the web.
By "annotated images", I mean that, using the PAIS authoring tool, an author can add information to an image. This annotation information is stored independently from the image data, and combined with the image data at display time, which allowing both the author and the user to control how it is viewed. By "linked image collection", I mean that, in addition to annotations, an author can create a navigation structure that will allow the user to easily move from image to image. Finally, by "viewable over the web", I mean that access to the images is through a simple HTML web page, which downloads the image viewing client software.
PAIS is also a research tool to let me experiment with different models of image annotation, as well as a development tool to track the development of the Image Markup Language (IML).
An "annotated image" is the combination of an "annotation file" and its associated image.
The annotation file contains a pointer (relative URL) to the image file, and is considered by PAIS to be the primary object (as opposed to the image being the primary object). These files are created using the authoring tool included with this system. A single image may be referenced by many annotation files.
Annotations are presently text comments pertaining to a region of an image.
The present annotation model is taken directly from that of the UW Digital Anatomist, and is completely compatible with its existing "Frame Format 2.0" annotation schema of those online, interactive atlases. The FF2 model has been implemented as an XML Data Type Definition document, defining the Image Markup Language 1.0 (IML 1.0).
The annotation metaphor is that of gross dissection, with a label connected by a string to a pin, which is stuck in some "structure" in the image. More specifically, each annotation consists of a text "label", a colored "string" connecting the label with a "pin" on the image, and an "outline" - a closed polygon "drawn" over some portion of the image.
Annotations may be either of type "STRUCT", which are regions tied to a text label, or "CONTROL", which are regions linked to another annotation file, similar to "hot" areas on an image map.
Image Markup Language (IML) is an XML syntax for expressing spatially referenced image annotations and other metadata. You can read more about it here.
The IML 1.0 model supports a single text string containing an image caption, and a series of individual image annotations referring to regions of the image.
Implemented in XML, the annotation files are just text files, marked up with a bunch of tag names contained within angle brackets. You can examine the format of these files with a text editor, though you might want to get a sense of what constitutes valid syntax before you greatly change them outside the comfort of the annotation editor.
The next IML revision will substantially extend the current annotation model to allow a richer expression of the annotation concept, and will be continue to be expressed as an XML schema At some point soon, I intend to switch from DTDs to the formal XML Schema syntax for the constraint document.
An image collection or atlas is simply a collection of annotation files and the images to which they refer. Since the file references are relative URLs, both images and annotations files may be arbitrarily organized.
To make an image collection browsable, you need only add a few files which comprise the image client, and put the whole folder in a location accessible by your web server of choice. No special server scripts or programs need to be installed.
The present client looks for an initial annotation file called "index.iml". Any other annotation files must be either directly or indirectly linked though "CONTROL" annotations to be reachable by the client. This restriction will also be relaxed in a subsequent version of the client.
There are two good ways to see PAIS in action.
You can look at the demo page for the IML1.0 version, and see a sample image collection (an anatomic atlas of the knee). If your browser does not have a Java 2 plug-in, you will be asked for permission to down load it from the Sun Microsystems site in either a fully automated (IE), semi-automated (Netscape on Windows), or manual (Netscape on Linux) fashion.
Second, you can download the image collection components yourself, and publish either a different sample collection, or build your own.
How do you do this?
Look at the PAIS Components section to see what components comprise a publishable image collection, and the Requrements section to make sure you've got the both the client and server requirements covered.. After the Download section, the Step-by-step section contains brief instructions for installing the components.
PAIS consists of a client components, test atlas components, and its authoring components, all of which are packaged together in a sample atlas download
The server side consists of a directory or folder whose files are served by a standard web server. This directory contains the images and annotation files that comprise the published image collection, and at least the following five files:
index.html - a sample HTML home page for an atlas. This does nothing more than jump to the Japplet page.
JappletAppl.PlugIn1.3.html - a sample HTML home page for an atlas. This version was produced with Sun's HTML Converter 1.3, and based on a dynamic check of browser versions, tries to offer a download of the Swing Plug-In, if needed.
IA.jar - a java archive file containing the client viewer applet.
jdom-b4.jar and xerxes.jar - XML support files
DemoPAIS.IML1.zip - contains a simple set of test images and annotations, as well as the executable versions of the client and authoring tools. See "Getting Started", below.
You can view the test image collection online here, if you want to know what it should look like.
The authoring side consists of a Java 2 environment, which you download from Sun, and the following four files. The XML support files are also used by the Client.
AI.jar - a java archive file containing the authoring application.
jdom-b4.jar and xerxes.jar - XML support files
AnnoteImage - a shortcut containing the command line "java -cp AI.jar;jdom-b4.jar;xerxes.jar AnnoteImage".
You'll need to have Java 2 installed for the authoring tool to work. See the requirements.
This section walks you through the requirements for each component, how and what to download, and a step-by-step approach to getting things set up.
The basic requirements for the server, client, and authoring tool are fairly straightforward. Since the client does the work of dynamically rendering the image and its annotations, the load on the server is quite small.
To serve the images, you need only a computer with a standard web server. There are no special language support or scripting requirements for the server. So far, the sample atlas has been tested on Microsoft personal Web server, Microsoft IIS, and the Apache server under Linux. Works fine. The served folders or directories do, however, have to be browsable.
To view the annotated images, you need a client computer with a web browser which either includes Swing for Java, or which can use a Swing Plug-in. Windows is supported with at least IE 5 and Netscape 4.7, I'm pretty sure Netscape 4.5 and IE 4 will work, and maybe earlier versions too. (But, you really should upgrade your browser!) The details of how the plug-in works are to be found on the Sun Java Plug-In page.
The plug-in will load automatically (after obtaining permission) with Internet Explorer and somewhat automatically with Netscape on Windows.
For the authoring code to work, you will need to have Sun's Java Software Development Kit (SDK) or Java Runtime Environment (JRE) installed on your computer. Java 1.1 will work, but you might as well install Java 2. You can find them, and a whole lot of other Java stuff, here for Windows, Solaris, and Linux. I'll get around to posting Mac and Linux instructions soon, I hope.
Let me know if you have problems with this. I have started working on an installer version for Windows, but it is lower on my list than features at the present. If no one can figure out how to run this stuff, that will change. Please let me know.
NOTE: Remember, both source and sample atlas (executables) are about to be replaced with new versions!!! (see note at top of page) Download at your own risk!
The holy grail - tucked away in the middle of the page. Hope there were enough links to get you here easily! This is where you can download a sample atlas, bundled with the client applet and authoring application. (download sample atlas with executables 930 Kbytes).
It is also where you can download the client applet source code (Java 2 - 36 Kbytes) and the authoring application source code (Java 1.1 - 92 Kbytes).
You might want to print a copy of the Slice of Life 2000 conference poster, which has an overview of the steps to create an annotated image collection.
Download the sample atlas. Unzip it. Move the entire folder where your web server expects to see it (C:\Inetpub\wwwroot under Windows 2000 Professional, for example). Access it from your browser by a url like http://yourserver.edu/DemoIML1.
You should be asked about downloading the Java 1.3 Plug-In, and then you should see the client run with a trial image collection of 5 images.
You can view the same test image collection online here, if you want to know what it should look like.
If so, you're all set to make an image collection of your own. Make a copy of the DemoPAIS.IML1 folder called, for instance MyImages, also located where the web server can see it.
If you don't already have Java, go to http://java.sun.com/j2se, and download either the Java 1.3 standard edition or runtime environment.. Install it.
Go back to your MyImages folder. You should be able to start the annotation tool by clicking on the AnnoteImage shortcut.
Now close the application, and drag some GIF or JPEG images of your choosing into the folder. Open the AnnoteImage application again, and create annotations as you desire. Look to the existing annotations for examples. As with all early release authors, I promise a tutorial will be coming soon...
- If this all looks like Greek to you, and you really want to get an image collection going, show it to a web administrator at your site - it really should not be too hard to get running. If it is, let me know and I'll figure out why.
Sorry to have misled you, but the tutorial is another one of those back burner projects, at least until someone start asking for.
Until then, download the sample image collection, as above, look at the .iml files with the authoring tool, and give it a try....
Now. Functional development versions of the authoring application and client image viewing applet are available here. They will get much better, I promise! Both executable (jar files and sample image collection folder) and source downloads are available
Of course there's a mailing list for PAIS. You can sign yourself up here: http://ohana.org/mailman/listinfo/pais. Don't worry, its an extremely low volume list!
See the content frame to the left, near the top.
Release Date Description 7/2000 minor updates to software, big updates to web pages - version shown at Slice of Life/CHES conference, Salt Lake City, June 28, 2000. 6/2000 0.9beta, uses Image Markup Language 1.0,
which I've submitted to the XML.org registry.5/2000 0.7beta, still "IA7" client code, but updates to home page. 3/2000
Original PAIS, using Digital Anatomist Frame Format
The images in the test image collection are copyrighted, as noted above and in the collection itself. The code is available under the GNU public license (GPL). For more information, see: the GNU Project. The code was developed while I was pursuing a fellowship funded by the National Library of Medicine, and while I was employed by the University of Washington. Those two entities may or may not have policies which are either more or less restrictive than the GPL, and those policies may or may not apply to this code. I'll try to find out.
is welcomed. Please contact me either at lober@u.washington.edu, or via the mailing list.
Hope you find this useful,
Bill