Chapter 1. Architecture Overview

Chapter 1. Architecture Overview
Prev	Part I. Lire Architecture	Next

Table of Contents

Lire's Design Patterns
Log File Normalisation
Log Analysis
Report Generation
Report Formatting and Other Post-Processing
Going Further

From a developer's point of view, Lire intends to be the

universal log analysis framework. To this end, it provides a reliable, complete, framework upon which to build log analysis and reporting solution. Lire, the tool, is a proof of the versality and extendability of the framework as it is able to produce reports for many of the services that run in today's heterogeneous networks in a variety of output formats.

As a framework, Lire is the best choice to replace all those home-grown scripts developed to produce reports from all the log files from the little-known products or custom-developed programs that run on your system. Leveraging Lire framework will make those scripts a lot more versatile while not being really more complicated to develop. It will be easier to add new reports or to support multiple report formats.

Figure 1.1. Log Processing in the Lire's Framework

The Lire's framework divides log analysis in four different processes. The figure Figure 1.1 shows those four processes:

Log Normalisation. The first process normalise logs from different products into a generic format that can be shared by all products that have similar functionality. For example, log files from products as different as Apache and Microsoft Internet Information Server will be transformed into an identical format.
Log Analysis. In the analysis process, other information is created, inferred or extracted from the normalised data. For example, an anlyser in the www superservice infers the browser used by the client from the referrer information.
Report Generation. The third process generates a report from the normalised and analysed data. This process is done by a generic report engine that computes the report based on specifications describing what and how the information should appear in the report. The report is generated in a generic XML format.
Report Post-processing and Formatting. The last process converts the generic report into a specific format like ASCII, PDF, HTML but other kind of post-processing (like charts generation) can also be accomplished in this stage.

Before going into a more detailed description of each of these procesesses, we'll introduce some of the common design's patterns that you'll find throughout the Lire's framework.

Lire's Design Patterns

At the center of each of these processes is an XML based file format. Having things specified in data files makes it easier to extend. For example, the reports are built using a generic report builder which finds the instructions on how to build the reports in XML files. So this makes it easy to add new information to a report: you just have to write an XML file. The fact that there are a lot of tools to process XML files is also an interesting aspect. For example, emacs lovers will appreciate the help that its psgml module gives them in writing report specifications.

Another important aspects is that we tried to interoperate and to build upon other standards while defining our XML formats . The best illustration of this is that in all the XML file formats that Lire use, a DocBook subset is used for all elements related to narrative descriptions.

Another common aspect you'll encounter is that each of these processes and XML file formats come with an API to manipulate them, making it easy to add functionalities at each processing stage. APIs are also a good thing because, even if in theory an open file format somewhat constitutes an API, having libraries that provide convenient access to the file formats makes it a lot easier to write new components providing new functionalities.