Image analysis and metrics for histology slide processing. I collaborate with Carolyn Salafia (Placental Analytics, LLC) and a masters student, Kristy Thomas, on segmentation and geometric measures derived from slide images for understanding properties of the human placenta.

The goal of this work is to understand how properties of the circulatory system of the infant present in the placenta can be measured through two dimensional slice images. Furthermore, we would like to understand what geometric measurements can be derived from the images that quantify properties known to impact the health of the child.

My work here is currently on reliable segmentation of the histology slide images to isolate various meaningful features. I have also contributed to the development of cost metrics to quantify the transport properties of the placenta vascular structure from surface photographs of large scale blood vessels.

Image registration for quantitative analysis of c. Elegans neural properties. Shawn Lockery of the UO Biology department is a neuroscientist who studies these little nematodes to understand the function of the worm brain. As a part of this work, Serge Faumont (a postdoc in Shawn's lab) and others have built aparatus to track and record video of the worm movement under a microscope. The goal is to watch the change in intensity of a specific region of the movie frames over time. Unfortunately, although the tracker keeps the worm head in the frame as it moves, it does not compensate for the rotation of the worm as it moves about.

I have developed some code based on the NLM Insight Toolkit (ITK) that performs this registration of the worm over the movie sequences to fix the worm head in place to produce a movie for quantitative analysis that is as though the camera both tracked and rotated with the worm as it moved. An example of this can be found here(.MOV), with the original movie before registration shown here.

I have been a participant in the DOE Common Component Architecture (CCA) project for the last 7+ years, and am currently responsible for the CCA Onramp tools. Onramp is work being performed by myself and a PhD student, Geoff Hulette, and aims to provide automation tools for assisting scientific programmers with migrating legacy C, C++ and Fortran codes to a component-based environment. Onramp operates in conjunction with the BOCCA tools that are a core part of the CCA toolkit.

The Open Fortran Parser (OFP) project is an effort led by the Los Alamos National Laboratory with contributors from many institutions from outside the lab, including UO. I have been a participant since the start of the project while I was still at LANL, and have assisted in the design of the software, maintainence of the sourceforge site, and experimentation with a Standard ML-based syntax tree tool.

Beyond providing an open parser (and ultimately a more complete compiler front-end) compliant with the Fortran 2008 specification, we plan to use this infrastructure to develop code transformation techniques for retargeting codes to take advantage of modern processor architectures. An example of this was demonstrated by the OFP project leader, Craig Rasmussen, in which a portion of a Fortran subroutine was transformed into C code encapsulated by libraries in order to run on the IBM Cell processor.

[OFP Home Page]

A common activity in numerical computing is the optimization of a set of parameters to minimize the value of some objective function. In simulation-based science, this objective function is often a metric derived from the evaluation of a complex model. In this project, the model was a standard 2D Navier-Stokes fluid dynamics simulation where the objective function was a metric comparing the simulation output to an experimentally obtained data set such as the density of the fluid in a spatial region. The interesting feature of our problem was that the time required to evaluate the objective function was a function of the physical and solver parameters fed into the model. Many different runtimes would be observed for different parameter sets that yielded equivalent results under the metric. This variance could be accounted for by examining parameters related to solver convergence constraints, model fidelity and resolution.

My PhD work was focused on quantifying the effect of operating system interference on parallel programs, and using trace-driven simulations to understand the effect of varying levels of interference on observed runtime message passing patterns from real application runs. My work was more of a methodology development than a tool production effort. The work had two primary focuses. First, on the quantification of interference through the development and data analysis of the Fixed Time Quantum (FTQ) microbenchmark. Second, I looked at trace driven simulations to understand the amount of interference an application could tolerate before suffering increase runtimes, and when runtimes increased, how the increase related to the interference level. The method of trace driven modeling was not unique, and closely tracked with methods developed as part of the Barcelona supercomputing center's Dimemas tool, but contributed a novel application of the technique in the specific context of OS interference understanding.

FTQ was first published in Cluster 2004, and has been subsequently used by IBM, LANL, LLNL, Cray, and others in the study of the performance characteristics of modern HPC operating systems. FTQ was considered for inclusion in the LLNL Sequoia platform procurement benchmark suite, but recently was replaced with it's close cousin, the Fixed Work Quantum microbenchmark. The reason for this change is that FWQ is ideal for measuring interference for the purpose of building comparative metrics. FTQ was created to build diagnostic metrics - essentially, when FWQ reveals unacceptable levels of interference, FTQ can be used to generate data from which diagnostic decisions can be made to identify and fix the source of the interference.

This work was published in Cluster 2004 and IPDPS 2006, and was a collaboration with many people, including Ron Minnich, my PhD advisor David Bader (now at GATech), Vaddadi Chandu, and others. A deeper study of the analysis of FTQ and FWQ data is pending publication in the not so distant future. I do not anticipate active development of this work beyond these papers though, hence the placement of this project in my "Previous work". The figure to the left is a message passing graph used for noise propagation analysis and included in the IPDPS 2006 paper (and my dissertation). This work was funded by the DOE FastOS program.

From 1999 onward through 2005 or so, I worked on the LANL Cluster Research Team on a clustering suite known as Clustermatic. Supermon was a component of that suite, and provided monitoring for large scale clusters with the primary goal being minimization of overhead. Monitoring is a tricky business at scale for a couple reasons. First, anything at scale is fairly difficult - especially aggregating data from thousands of data sources quite frequently. Second, it is well known in the high performance computing world that parallel applications that use synchronous operations (such as synchronous paired communications or collective operations) amplify small isolated performance perturbations in a way that slows down the entire application. This is known as Operating System Noise (or OS Interference). Given that Supermon shares both processing and network resources with applications (we did not assume the presence of a secondary, out of band monitoring network), we sought to reduce the overhead required to perform monitoring.

The project has been fairly successful, although the lack of any dedicated people who could spend all of their time on the project meant that we couldn't keep up with the competition. Regardless, supermon has proven to be useful for the data gathering layer for clusters even today, with the interfaces and glitz provided by tools like Ganglia acting as the user interface. Development of supermon is still active and contributions are being provided primarily by Hewlett-Packard developers since HP includes supermon as part of one of their clustering solutions. (References to Supermon on HP's site.)

Clustermatic (including supermon) was one of the winners of a 2004 R+D 100 award. Ron Minnich was our fearless team leader, and original creator of Supermon in the mid-1990s.

[Supermon Home Page]

Retargeting existing code for accelerator hardware (such as FPGAs) requires analysis of the code to identify regions that have properties that can be exploited for high performance by the accelerator. Arbitrary code mapped to an accelerator is highly unlikely to perform well, if at all. This work was an early exploration of tools for analyzing the observed runtime properties of scientific codes to aid in identifying candidate regions for acceleration.

This is a project I spent some time on while still at the LANL ACL with Maya Gokhale. The abstract and slides for the work are available here. I contributed the OpMix tool to the project and performed the mix analysis on the simple 2D CFD code used as an example. This work was presented at LACSI 2006, HPEC 2006 and PACT 2006.

[Abstract] [Presentation]

One of the more fun projects I worked on while at LANL was assisting a large team of researchers investigating the viability of muons originating from cosmic rays for the detection of nuclear materials in vehicles or containers entering the country. The basic idea is that muons, being a charged particle, will scatter differently depending on the material that they pass through. Higher Z materials will cause higher scattering angles. Due to the low flux of muons from space, it is possible to track them individually through a series of detectors to see how they interact with materials that they pass through. Unlike active methods (such as X-rays), muons are present everywhere and are just part of the natural background.

My role on this project was assisting in deploying a GEANT-based physics model to use large scale parallel systems at LANL for generating data to develop, test, and validate detection algorithms that used scattering information from detectors to infer the presence of materials inside vehicles. My contributions were in the optimization of the code (which, at one point even involved analyzing the numerical algorithms and applying trigonometric identities to yield equivalent results with lower floating point operation counts, beyond the capabilities of optimizations that compilers provided), and the execution and management of many thousands of runs on the LANL Pink cluster that I helped construct.

A picture of myself, Erik Hendriks, Sung-Eun Choi, and Ron Minnich in front of a third of Pink is shown on the left shortly after the machine arrived and was assembled. The four of us, plus Greg Watson (not pictured) set up Pink, a 1023 node, 2046 processor cluster, in 2003. While not big currently, at the time it was a very, very capable machine, and produced massive amounts of useful data for the muon project before it was handed off from my team to the larger lab user community.

To my knowledge, this project is no longer as active as it was when I worked on it in the 2004-2006 timeframe. More information on the project can be found on the LANL web site here. A couple of papers resulted from this work that can be found in my CV.