Reading and Writing Analytics

We have been looking at capturing data on both reading and writing with an eye towards providing an online intervention for those who struggle with either or both. For reading, we have built on top of the CampusReader project. For writing, we have just begun to explore what we call CampusWriter. See below for references.

Gutierrez, F.; Dou, D.; Fickas, S. & Griffiths, G. Providing Grades and Feedback for Student Summaries by Ontology-based Information Extraction, Proceedings of the 21st ACM Conference on Information and Knowledge Management (CIKM), 2012

Gutierrez, F.; Dou, D.; Fickas, S.; Martini, A. & Zong, H. Hybrid Ontology-based Information Extraction for Automated Text Grading, Proceedings of the 12th IEEE International Conference on Machine Learning and Applications (ICMLA), 2013

Gutierrez, F.; Dou, D.; Fickas, S. & Griffiths, G. Online reasoning for ontology based error detection in text, Proceedings of the 13th International Conference on Ontologies, DataBases, and Applications of Semantics (ODBASE), 2014

Notes: Our goal is to check reading comprehension: does a student understand what they have just read? These papers focus on finding logical errors in student summaries of text they have read. Approaches that rely on LSA-style analysis will not work here. As an alternative, we are exploring using an Ontology as the ground truth and parsing student statements into a form we can compare against the Ontology. We have also started to look at "error rules" for more direct pattern matching. Finally, we are looking at hybrid approaches that mix OBIE (ontology-based information extraction) and rule-based information extraction.

Ledbetter, A. K., Fickas, S., and Sohlberg, M. M. (2014, March). Evaluating the Use of a Computerized Writing Log for Assessing the Writing Process in Individuals with Acquired Brain Injury. Poster session at the International Brain Injury Association, Tenth World Congress, San Francisco, CA. (pdf slides)

Ledbetter, A. K., Fickas, S., and Sohlberg, M. M. (2014, November). Evaluating the Use of a Computerized Writing Log for Assessing the Writing Process: A Pilot Study. Presentation at the American Speech-Language Hearing Association Convention in Orlando, FL.

Notes: These presentations are our fledgling attempt to get a handle on the process of writing. Research to date has instead focused on analyzing the product. We believe that process information can be a complement. Although the research reported is on a small n of 3 matched pairs, it points to some interesting directions for future work; patterns in the data are suggestive of (1) potential for process data to (1) predict product outcomes and (2) discriminate typical from struggling writers.

The Software Pharmacy

This work attempts to integrate personal requirements engineering with software product lines.

Sutcliffe, A., Fickas, S., Sohlberg, M, Personal and Contextual Requirements Engineering, 13th IEEE International Conference on Requirements Engineering, Paris, September 2005 (pdf). Awarded best paper at the 13th IEEE International Conference on Requirements Engineering (RE'05).

Sutcliffe, A., Fickas, S., Sohlberg, M., PC-RE: a method for personal and contextual requirements engineering with some experience, Requirements Engineering, Mar 2006, Pages 1 - 17 (pdf).

Notes: a conference paper and expanded journal paper that layout the Software Pharmacy framework. They describe the basic model we use today.

Fickas, S., Sohlberg, M., Hung, P., Route-following assistance for travelers with cognitive impairments: A comparison of four prompt modes, Int. J. Human-Computer Studies, Volume 66, Issue 12, December 2008, Pages 876-888 (pdf)

Lemoncello, R., Sohlberg, M.M., & Fickas, S. (2010). How best to orient travelers with acquired brain injury: A comparison of three directional prompts. Brain Injury, 24, 541-549.

Lemoncello, R., Sohlberg, M.M., & Fickas, S. (2010). When directions fail: Investigation of getting lost behavior in adults with acquired brain injury. Brain Injury, 24, 550-559

Fickas, S., Lemoncello, R., Sohlberg, M., Albin, R., Harn, B., Television Prompting of In-Home Rehabilitation Exercises: Promising Results, In Workshop on User Models for Motivational Systems: the affective and the rational routes to persuasion, part of Conference on User Modeling, Adaptation and Personalization, June 2010 (pdf).

Notes: We used our Software Pharmacy framework to quickly build the necessary artifacts needed for these studies. The last, in particular, required extensive personalization of television interfaces to match divergent needs of post-stroke patients.

Sohlberg, M. M., Fickas, S., & Griffiths, G.G., "Reading comprehension strategies delivered via tablet for individuals with acquired brain injury", (2011). American Speech Language and Hearing Association Annual Conference

Griffiths, G.G. & Sohlberg, M. M, "Same complaint, different profile: An examination of reading comprehension and retention deficits of adults with ABI", (2012). Journal of International Neuropsychological Society

Sohlberg, M.M., Griffiths, G.G., & Fickas, S, "The effect of electronically delivered strategies on reading after mild-moderate acquired brain injury", (in review). American Journal of Speech-Language Pathology

Notes: This set of papers is focused on our CampusReader project. We heavily use the Software Pharmacy framework in personalizing our eReader to different student impairments.

Clinical Requirements Engineering

I became interested in this area when I began to work with researchers doing clinical science. The clinical perspective is all about personalized "treatment" or solutions. I'll list a set of papers with brief notes on each that give a picture of my work in this area. I need to acknowledge the help of the RE community, and in particular the IFIP 2.9 Requirements Engineering working group, in honing my ideas on the topic. I also need to recognize my good colleague McKay Sohlberg (Cognitive Rehabilitation). McKay, and her colleagues have been instrumental in opening this line of research for me.

Fickas, S. Clinical Requirements Engineering. Invited paper at the 27th International Conference on Software Engineering (Extending the Discipline track), St. Louis, May 2005 (pdf).

Notes: This was a first attempt to come to grips with RE in the clinical sciences.

Fickas, S., Robinson, W., Sohlberg, M, The Role of Deferred Requirements in a Longitudinal Study of Email, 13th IEEE International Conference on Requirements Engineering, Paris, September 2005 (pdf).

Notes: an interesting concept used in clinical science is that of "goal attainment scale". The general idea is that you generate a graduated set of requirements (goals) that a user can work towards. A user may only be able to meet the minimal set now. But over time, they may be able to incorporate more ambitious goals, e.g., as their condition improves. This paper brings the idea into an RE context through the notion of deferred requirements, ones not relevant today but possibly relevant as things change.

Requirements Monitoring

You typically generate a set of requirements for a system under a set of constraints. Assume those constraints change once the system is in place. It might be prudent to monitor the system to see if the requirements are still accurate. My good colleague, Martin Feather, and I came up with this general approach in the mid-90s and reported on it in a 1995 paper. Since then, there have been some very nice tools built to support the requirements monitoring process. And the idea still seems to be alive in the area of self-adaptive systems. My personal work in the area is more linked to the goal-attainment-scale concept, and monitoring for opportunities to introduce more ambitious requirements (see the Personalized and Contextual Requirements Engineering tab above).

Fickas, S., Feather, M.S., Requirements Monitoring in Dynamic Environments, Proceedings of the Second IEEE International Symposium on Requirements Engineering, IEEE Computer Society Press, York, England, March 1995 (pdf).

Notes: Martin and I are quite honored that this paper was awarded "Most Influential Paper from RE'95" at the 13th IEEE International Conference on Requirements Engineering (RE'05).

Feather M, S. Fickas, A. van Lamsweerde, C. Ponsard , Reconciling System Requirements and Runtime Behavior, Proceedings of IWSSD'98: 9th International Workshop on Software Specification and Design, Ise-Shima (Japan), April 16-18 1998, IEEE Computer Society Press (April 1998) (pdf).

Notes: this paper links requirements monitoring with GORE (Goal-Oriented Requirements Engineering).

My colleague, Bill Robinson, has a very nice requirements monitoring tool called ReqMon. It is free to download and use. Bill has an extensive set of test-cases described. See his web page: ReqMon Home Page.

Operational Envelopes

The base conjecture of this project is that engineered artifacts will not work dependably in every possible environment. When a system is designed, cost/benefit analysis will be applied to determine the likely environmental conditions it will face, and then design and implementation can concentrate on those subset of conditions. This results in an operational envelope for the system: inside the envelope, the system is dependable; outside the envelope, dependability bets are off.

I really like this project. It started with a small study with NASA/AMES (see paper below). It then blossomed into a larger project looking at the operational envelope of the Mars Rover: ROPE project. It currently now sits on the backburner.