Lydia/TextMap Publications

Websites presenting Lydia analysis include:

• TextMap identifies temporal and geographic trends in entities such as people, places, and things by analyzing roughly 500 daily English language newspapers.
• TextMed identifies relationships between medical and biological entities through analysis of PubMed/Medline abstracts.
• TextBlog identifies trends and entity relationships through large-scale analysis of blogs.
• TextBiz uses random-walk models to generate a probability distribution on the future prices for all NASDAQ, NYSE, and AMEX stocks. We soon should add analysis of business news sources.

Research papers describing our work are:

• Lydia: A System for Large-Scale News Analysis by L. Lloyd, D. Kechagias, and S. Skiena, 12th Symp. of String Processing and Information Retrieval, (SPIRE '05), Lecture Notes in Computer Science, 3772 (2005) 161-166 provides an overview of the architecture of the Lydia system as of May 2005.
• Question Answering with Lydia by J. Kil, L. Lloyd, and S. Skiena, 14th Text REtrieval Conference (TREC 2005), NIST Gaithersburg MD, November 15-18, 2005 describes an extension to Lydia for answering factoid, list, and open-ended English-language questions.
• Newspapers vs. Blogs: Who Gets the Scoop? by L. Lloyd, P. Kaulgud, and S. Skiena, AAAI Symp. Computational Approaches to Analysing Weblogs (AAAI-CAAW 2006), Stanford University, March 27-29, 2006 provides an comparison of entity frequencies between blogs and more formal news sources.
• Identifying co-referential Names Across Large Corpora by L. Lloyd, A. Mehler, and S. Skiena, Proc. Combinatorial Pattern Matching (CPM 2006) discusses our method for identifying synonym sets of entities.
• Spatial Analysis of News Sources, by A. Mehler, Y. Bao, X. Li, Y. Wang, and S. Skiena, IEEE Trans. Visualization and Computer Graphics 12 (2006) 765-772 discusses our ``heatmap'' analysis.
• Large-Scale Sentiment Analysis for News and Blogs (with N. Godbole and M. Srinivasaiah). Int. Conf. on Weblogs and Social Media (ICWSM 2007), Denver CO, March 26-28, 2007. Also see our system demonstration description.
• Concordance-Based Entity-Oriented Search (with M. Bautin) IEEE/ACM Web Intelligence (WI-07), Silicon Valley CA, November 2-5, 2007.