Log4Moz Explicit Logging
The gloda code has Log4Moz logging statements spread throughout. The following preferences are supported to control how this gets logged:
- mailnews.database.global.logging.console - Should warning and error messages be logged to the error console window (accessible via Tools... Error Console)
- mailnews.database.global.logging.dump - Should all messages be logged to stdout by the dump() command? This does require that browser.dom.window.dump.enabled is set to true.
- mailnews.database.global.logging.net - Should we look for a /tmp/chainsaw.ptr file that tells us a hostname and port to connect to to send XML formatted log records to. The file should be of the form "hostname:port" like "localhost:1234".
Gloda will run faster if only error console logging is enabled; some of the debug-level log statements and their arguments are entirely predicated on the gloda logging level.
SQLite Query EXPLAINations
The gloda datastore will write a JSON file with the EXPLAINed results of all its SQL queries to the path found in the preferencemailnews.database.global.datastore.explainToPath. This preference is observed so that setting it during runtime will cause it to create the file and begin explaining all subequently created queries. Clearing/changing the preference closes out the current file and potentially opens a new one.
Gloda unit tests will automatically set the preference to the value of the environment variable GLODA_DATASTORE_EXPLAIN_TO_PATH if set.
See the following blog post for more information and tools on the subject.
SQLite Queries you can run against the database
An important thing to know about the gloda database is that because we use a custom tokenizer ("mozporter"), running the SQLite3 command line tool or any other tool against global-messages-db.sqlite will fail to be able to treat the fulltext tables as usable FTS3 tables. This does not matter for poking and prodding because FTS3 actually uses multiple backing tables under the hood, and we can directly poke at them. Specifically, in the case of messages, our FTS3 virtual table is named "messagesText" and you won't be able to poke it, but all of its nougaty goodness is stored in "messagesText_content", "messagesText_segdir", and "conversationsText_segments". The segment tables just store giant BLOBs that are not human readable, but "messagesText_content" has the acutal full text that we care about.
Fulltext message size distribution
The following query will tally the fulltext content size distributions of messages in (rounded-down) kilobytes. Enter it into the SQLite3 commandline tool against global-messages-db.sqlite. It will take a bit of time. Note that bug 543737 will cap the body sizes making this somewhat less interesting.
select (length(c0subject) + length(c1body) + length(c2attachmentNames) + length(c3author) + length(c4recipients))/ 1024 as kilos, count(*) from messagesText_content group by kilos;
The results will look something like this:
|158
0|74125
1|33622
2|9917
3|5026
4|2126
5|1252
6|862
7|658
8|666
9|566
10|372
11|378
12|326
13|183
14|179
15|124
16|108
17|79
18|64
19|51
20|62
21|34
22|26
23|19
24|26
25|16
26|19
27|7
28|4
29|4
30|13
31|6
32|11
33|7
34|6
35|3
37|3
38|2
40|3
41|6
42|2
43|4
44|4
45|7
46|2
47|1
48|2
49|1
50|1
53|2
58|1
61|1
62|3
63|2
65|2
81|4
83|2
88|2
89|2
96|2
101|2
108|2
109|4
116|2
118|2
254|2
303|2