2004

January

1^st January 2004 Happy New Year.

February

7-8^th February 2004 It appears the Ruby regular expression engine was replaced last night in Debian Sid (I update daily)  ³³⁶  which resulted in SiSU to sputtering and made necessary an unexpected code updating of SiSU... no more sputtering SiSU is back in alignment, and (taking into account the possibility of a few remaining bugs as yet undetected) under control.

Around this time I saw AniMatrix ... my favourite movies have remained stable for a while, this perhaps makes the list. The Usual Suspects (clever, fun), 12 Monkeys (science fiction, dark, disturbing, it works), Being John Malkovich (illogical and extremely clever, a quirky combination that works), The Matrix   ³³⁷ 

10^th February 2004 SiSU man page added (other forms of help are also provided).

25^th February 2004 Bought and read "RELAX NG" by Eric van der Vlist.  ³³⁸  Started "modelling" SiSU XML structures. Glad to have waited to look at XML.

March

4^th March 2004 Bought March issue of the Linux Journal, issue 119 for articles on "Standard Office Document Format: Oasis"  ³³⁹  and "Manipulating OOo Documents with Ruby".  ³⁴⁰  Glad to have waited this long to look at XML.

4^th March 2004 SiSU's online help improved, introduced SiSU help prompt (other forms of help are provided including a man page).

6^th March 2004 SiSU generates well formed XML, first tests passed. A way to go... Glad to have waited this long to look into generating XML. XML output is likely to be modified. Will add some semantic XML markup options to the SiSU's input/markup syntax... (e.g. titles, authors etc. for books cited within text). Bear in mind, the system is to be kept uncomplicated.

7^th March 2004 Added /darcs/  ³⁴¹  to Ratchet. Ratchet being a simple graphical front-end for repetitive backup and version control commit tasks.  ³⁴² 

18^th March 2004 Very easy to poll semantic data (Dublin Core info set) in SiSU marked up files to produce /RSS/ feeds, of for example all documents on a given subject, keyword, or by a given author, or whatever should be included in the feed. The feed is built based on your document set.

23^nd March 2004 A current thumbnail and screenshot, some help output from SiSU and ratchet in its present form (if you go for the screenshot, it may be necessary to scroll right to see ratchet):

ratchet, in the right corner of the screen, and some help output from SiSU

March / April 2004 Ralph Amissah - Workshop on Advanced Structured Trade Finance, focusing on Legal Documentation of Trade Finance in Africa organised by the African Export Import Bank, part of panel on "Opportunities and Challenges in Documenting Trade Finance Deals in Africa".

April

3^rd April 2004 Vim color syntax files, greys: slate  ³⁴³  or dusk, and shades (also gold)

9^th April 2004 ascii output... finally got round to producing acceptable stripped ascii output for SiSU

11^th April 2004 "Free Culture", by Lawrence Lessig  ³⁴⁴  (largely) marked up, a prelude to reading it (have not actually seen the book, it is not yet published in the UK). Marked up "Free For All", by Peter Wayner  ³⁴⁵  back in 2002 (and emailed it to the author). Will place both these books as output and markup examples (and will offer these versions to the authors).

"Free For All", Peter Wayner

"Free Culture", Lawrence Lessig

12^th April 2004 As am testing SiSU markup requirements, and suitability for basic literature text selected a text from the Gutenberg Project, and did a transform. The text chosen: "War and Peace" by Leo Tolstoy, (it came to mind as a largish volume) which I shamefully confess not to have read. The markup transforms required are trivial. Of interest, in this instance I am saved by having alternative modes for marking up endnotes... as can do a simple search and replace to make suitable for SiSU using the older endnote markup style.

"War and Peace", Leo Tolstoy

and the markup:  ³⁶³ 

‹http://www.jus.uio.no/sisu/sample/markup/war_and_peace.leo_tolstoy.sst›

‹http://www.jus.uio.no/sisu/sample/syntax/war_and_peace.leo_tolstoy.sst.html›

Part of help output from SiSU ...

help output from SiSU and in the right corner of the screen ratchet

16^th April 2004 Minor aesthetic improvements to ascii output. Some further cleanup of output required, but done with general presentation of ascii for now. (Should organise meta-data... semantic information at the end of the ascii file better). Starting to notice and read about several "smart ASCIIs", primarily for ascii to html conversion, reasonable looking ascii with minimal markup requirements for conversion to html (other formats) their goal.  ³⁶⁴  Might make sure at some future date that SiSU ascii output is some form of (existing) valid Smart ascii.

17^th April 2004 Bored with the old anagram, SiSU as "simple - information structuring unit", or "sisu is structuring unit"... how about "*semantic* and information structuring universe" ;-)  ³⁶⁵  well fun anyway, will work a bit on minimalistic semantic ideas, but as they say KISS!  ³⁶⁶  ok, "simple - information structuring universe" then.

24^th April 2004 Bugfix, SiSU once again permits footnotes/endnotes in headings.  ³⁶⁷ 

Got my hands on a new edition of "Guide to LaTeX" 4th Edition, by Kopka, H. and Daly, P.W., 2004 Addison Wesley, ISBN 0-321-17385-6.  ³⁶⁸ 

28-29^th April 2004 Having the new edition of Kopka and Daly gave me the excuse I needed to look at LaTeX output. I have used it surprisingly little, but it gave the motivation, and it is a good reference. Knowing that it is there provides some form of information management guarantee, I have the confidence to look at issues, knowing the book is on hand and is likely to provide support should I need it, most of the time it is not needed, but it would have been a waste of time to work on the issues if I had needed it. Lots of small, niggly outstanding issues sorted, things that irked me, like my indented text had too much space between paragraphs (book not needed), and the text within table columns did not wrap (book used). Still some cleaning to be done.

May

2^nd May 2004 A host of subtle improvements to LaTeX/pdf output, from page breaking to cover page.

6^th May 2004 /Ratchet's/ interface simplified by taking out /arch/ centricity. Have looked a bit more closely at /darcs/ as suggested by Ion's creator. /Ratchet/ is just a slapped together, custom built front-end for my version control and backup tasks, and it would not have been written if it had not been for my unfamiliarity with, and the relative complexity of /arch/ commands.  ³⁶⁹ 

ratchet on the right; the program text for ratchet in gvim (with my slate color scheme), on ion

wmi  ³⁷⁰  the "vim of window managers" spotted in a discussion on the #ion channel. Yet another window manager, this one I like lots, I feel at home here.

ratchet to the right; the program text for ratchet in gvim (with my slate color scheme), on wmi

Ion  ³⁷¹  is my preferred window manager..., but wmi is tempting.

7^th May 2004 /SiSU's/ syntax for the time being SiSU permits the other smart ascii markup style without curly brackets, for single words only. This leads to serious consideration of having multiple input syntaxes depending on document type, the current default suits a wide variety of documents from literature to law..., but a different subset of rules would be appropriate for technical writing.

Could not resist another screenshot, SiSU gVim programming work environment, and my /"slate"/ syntax highlighting color scheme  ³⁷² 

SiSU project files, gvim on wmi

SiSU project files, gvim on ion

First read about /glark/  ³⁷³  and switched to using it.  ³⁷⁴ 

June

5^th June 2004 Batch conversion to SiSU file format (text markup) from MS Word97, using wvWare  ³⁷⁵  and a some additional (Ruby) processing, works pretty well, does the initial processing. Potentially very useful...

17^th June 2004 Here is a souvenir from my encounter with an industry giant, IBM:  ³⁷⁶ 

"Ralph

Good to meet with you today, I was very impressed with your software.

[colleague's name]  ³⁷⁷  - in summary - Ralph has built an application that runs on linux and takes ASCII documents and pulls them apart in to the smallest constituent parts, storing them as XML, PDF and HTML, the HTML are hyperlinked up so the document can be browsed in its full form. the format and text data created is stored in a database.

This has potential in any place that needs the power of full text search whilst holding the structural concepts of the document i.e. legal, pharma, education, research.. which ones we need to figure out, ..."  ³⁷⁸ 

Special interest was expressed in the search implications of SiSU. To paraphrase, the company has document management systems dealing with hundreds of thousands of texts, these tell you which documents match your search criteria, but cannot inform you where within a text these matches were found without opening the documents. This, is incidentally a feature was implemented a number of years earlier, and part of the even earlier design goal, made possible by powerful implications of the simple concept - object numbering (object citation numbering, ocn).

A follow-up was posted a year later see note on abandoned patent process posted on 5^th June 2005   ³⁷⁹ 

20^th June 2004 A note on future work directions, where effort is likely to go. *(a)* Filters, for conversion between SiSU markup and the subset of Open Office "markup" that are supported, giving the possibility to do most document preparation in Open office and possibly further to use SiSU capabilities from Open Office documents. *(b)* Greater effort on the database frills. The backend structural logic and working functionality is in place, but there are some loose ends to tie, also there is added functionality that should be provided using SiSU that is implicit from an SQL DB's design (and the logical structure of tables that are in use). The front end, needs work. *(c)* [not to forget April 17] Semantic tags within text, (for use in for example in XML conversions) keep simple.

June-July 2004 Much work on html conversion tools.

July

10^th July 2004 Touched relational db  ³⁸⁰  frontend example, works as expected.

Simple search, results with files in which search found, and locations where found within files.

SiSU a frontend to postgresql

Simple search, results with files in which search found, and text object (paragraph or endnote) where found within files.

SiSU a frontend postgresql

Searches can be complex booleans.

14^th July 2004 (version control) Ended up making a console (tui) tool out of ratchet. With that the gui version becomes virtually redundant, as this is what I use instead.

Ratchet console/tui version

16^th July 2004 Syntax changes for headers and headings.  ³⁸¹  Filenames also shortened, as the extra digit is not used. The name changes restart the cvs ids.  ³⁸² 

August

9^th August Feature freeze on development branch for next generation of code.

August since abandoned provisional patent application, accompanying diagram   ³⁸³ 

30^th August - 30^th September 9^th 2004 Transition, a new generation of code. Primarily concerned with internals, and passing of information between objects. [lexmercatoria.org will continue to use previous generation code though]

September

8^th September 2004 Output directory rules simplified, output directory mirrors source directory "stub" name. You would place all associated documents in a directory of a name that signifies this association, and the output directory would be to a sub-directory of the same name in the web serving directory.

A similar rule is used for the sql database into which files are placed.

9^th September 2004

1. Directory file association, skins and special image management, made simpler.

The last part of the name of the work directory in which markup is being done, or rather from where SiSU is run in order to generate document output, is used in determining the sub-directory name for output files, that is created in the document output directory. This provides a rather easy way to associate documents e.g. of a given subject, or by owner.

  /www/docs
      /intellectual_property
      /arbitration
      /contract_law

  /www/docs
      /ralph
      /sisu

all are placed in their own directories within the directory structure created. Similar rules are used in the creation of sql type databases (though they can be overridden).

There are a couple of further associations with these directories.

Directory wide skins.

Directory specific images.

2. If there is a "directory skin", that is a skin of the same name as the directory, it is used in the generation of the documents within it, rather than the default skin, unless the document has a specific skin associated with it.

a. default skin (always available)

b. directory skin (precedence over default if exists)

c. document skin (takes precedence wherever document requests a specific skin)

Skins are defined in the document skin directory and if a directory association is desired a softlink made to the relevant skin. Skins (directory association auto load) auto load skin if a directory skin exists of same name as directory stub, (and there is no specific doc skin)

3. If the working directory has within it a sub-directory called image_local, the images within that directory are used for references to images, that are not part of the default site build.

14^th September 2004 Difficulty deciding on which file extension to use for sisu text files, currently ".r", ".s", ".t", ".rs" (and ".lm") with the digits 0-3 following the letter, [.er is dropped].

28^th September 2004 Further modification to file naming conventions, in future will use an underscore in preference to a dot for separation, e.g. eu_contract_principles_parts_1_to_3_2002.sst rather than eu.contract.principles.parts.1.to.3.2002.sst - old naming will also continue to exist.

October

1^st October 2004 The second edition of "Programming Ruby" aka Pickaxe II - my (first) copy arrives, a pdf.  ³⁸⁴  it is almost 4 years since the first edition of the Pickaxe,  ³⁸⁵  made my considerable Perl book library obsolete, overnight.  ³⁸⁶  The book original Pickaxe is pretty good, and available online, (the second edition is better still) but what was fantastic for me about the first Pickaxe which I was able to get my hands on in November 2000, was it was the gateway to Ruby (that I had been waiting for, since I had targeted evaluating the use of Ruby instead of Perl in February 2000, but had no wish to attempt to do that from scratch, i.e. without an English text on the subject).

Ruby takes over, November 2000

With Ruby's POLS,  ³⁸⁷  that type of book collecting became redundant... and the gap between idea and implementation narrowed... and code became more reliable/dependable/maintainable and more easily "flexed"... beauty, elegance... Happy coding.  ³⁸⁸ 

Around this time, I am not sure exactly when I also stumbled upon and read "A Hacker Manifesto" by McKenzie Wark, as I recall an excellent Marxist based analysis of the struggle over intellectual property rights in software development.

A Hacker Manifesto, McKenzie Wark

November

13^th November 2004 Ion window manager, style and the most powerful functionality  ³⁹⁰  and Screen  ³⁹¹  - screenshot of my current window manager and "term multiplexor", computer interface:

Ion3 - window 7 of 10

A bit contrived as I would seldom need that much clutter in the float window (left side of monitor), but it gives the general idea, and of how my Ion frames (for window 7) and term /screen(s)/ are set up.

December

8^th December 2004 Tentative steps on DB (postgresql) front-end, ie search form.

10^th December 2004 Finally got to look at a copy of the second edition of "The LaTeX Companion".  ³⁹²  Fantastic. I ordered a copy from Amazon on November 26^th, along with several other books, for some reason (could it be Christmas) Amazon is taking ages over posting my order! [finally shipped December 22^nd arrived December 30^th]

12^th December 2004 Front end for Postgresql  ³⁹³  database looking pretty good. Much work on the front end to the database, and now that I finally get round to looking at it, I wonder what took me so long. One thing is the fact that from the technology perspective it is the back-end database and its structure that really matter. Front-end(s) can then be provided in any number of ways, (though being what an end user sees is no less important).

Several options for output - select database to search, show results in index view (links to locations within text), show results with text, echo search in form, show what was searched, create and show a "canned url" for search, show available search fields. Also shows counters number of documents in which found and number of locations within documents where found. [could consider sorting by document with most occurrences of the search result].

Other things not done, a back-end issue with Postgresql, the default field/cell maximum size set by Postgresql for indexing on VARCHARS, is too small, would have to set that differently before production use.

I am fairly pleased with it. This sampling on a small database demonstrates functionality (which could be extended further with current back-end). I will prune the images later. Click on thumbnail for larger image:

SiSU search frontend to Postgresql the form when empty

SiSU search frontend to Postgresql available fields

SiSU search frontend to Postgresql author and text fields

SiSU search frontend to Postgresql title and text search, index view

SiSU search frontend to Postgresql title and text search, index view

SiSU search frontend to Postgresql title and regex text search, index view

SiSU search frontend to Postgresql simple search, index view

SiSU search frontend to Postgresql regex search, index view

SiSU search frontend to Postgresql regex search, text view

SiSU search frontend to Postgresql regex search, title War and Peace, text view

SiSU search frontend to Postgresql simple search, title restriction, text view

20^th December 2004 Change of revision control representation, using some recommendations from the RPA (Ruby Production Archive)  ³⁹⁴  team. Appears in Document Information under generated by, at this time SiSU Sabaki 0.1.0-6 of 2004w51/1 (20041220)  ³⁹⁵  Sabaki is a Japanese word that is used in the board game Go (Chinese Weichi, Korean Badok) to refer to shape, (the previous working name was Scribe).

31^st December 2004 Look at license and copyright notices. Problem, have opposed interests, one being my preference for the GPL  ³⁹⁶  with regard to program licensing, the other being to be able to maintain a standard for markup and for the object citation numbering system, and also have some recognition for what amounts to the results of my several years of research into electronic document publishing, as represented by SiSU and described in this document. I have ended up with a GPL license for the program, but claiming copyright over (i) the document markup syntax and meta-markup syntax, and more importantly, (ii) the object citation numbering system, a very simple idea that makes so much possible, from document format independent citation, to sql and other database search indexes, with citation references for matches. At present the license information reads as follows:

Generated by SiSU www.jus.uio.no/sisu/SiSU/ [ SiSU Sabaki 0.1.0-8 2004w51/4 ]

Using SiSU Standard markup and meta-markup syntax and SiSU's object citation numbering system © Ralph Amissah 1997, current 2004. All Rights Reserved.

SiSU is released under GPL 2 or later ( ‹http://www.gnu.org/copyleft/gpl.html› ) and is alternatively available under special license terms from Ralph Amissah.

I will take advice and recognise that it may have to be changed.

SiSU thanks Ruby and Gnu Debian Linux

Also start using more neutral icons. If I were not using Debian/Gnu/Linux Sid,  ³⁹⁷  and satisfied with the package manager, range of applications and social contract,  ³⁹⁸  I would probably be using one of the BSD's, Dragonfly  ³⁹⁹  or FreeBSD.  ⁴⁰⁰  Also I guess despite not judging the success of a project by its adoption, over promotion of a distro should not be placed as a disincentive for use on other Unix/Linux platforms.

SiSU Sabaki

End of year ToDo:

Release SiSU early 2005. Dual license, one of them "GPL 2 or later", other, reserved, proprietary.

Documentation. There is some, but the presentation is nowhere near as digestible as it should be.

Unicode UTF-8. The biggest single todo is Unicode processing. LaTeX and Postgresql support UTF-8 so that is what it is most likely to be. My excuse for not having looked at it yet ... need to date, and not having configured my environment for it. I do however recognise this as a need.

Redo sqlite module ... one of the more interesting to have. Get the Sqlite module working again. Similar to the Postgresql module, fell out of maintenance, when I found Sqlite to be a bit of a pain to install on Debian, (and was prioritising Postgresql), once upon a time the modules were in sync, and I hope to have them that way again someday.

Code cleaning. Much code cleaning ... this project has developed over several years, and there have been many changes in how things are done, without rigorous removal of dead code.

Simplify Installation, and test across other Unix and Gnu/Linux platforms. (Mac OSX too? I wonder...)

Object Citation Numbering for non-substantive objects. OCN is currently done only for substantive text and other objects (such as images), a secondary numbering will eventually be implemented for non-substantive items.

Images. Decide what to do with images and tables in XML and in relational database.

Marshalled/PStored Metaverse. As an alternative (not replacement) to the current ordinary text based SiSU meta-markup state.

Additional Syntax hi-lighters. The current syntax hi-lighter, and folds are for vim. Additional syntax highlighters for SiSU markup would be extremely welcome, they don't need to be as complete as the vim highligter. Emacs would obviously be nice, but the ruby editors, and less geeky editors are of much interest. Not sure that I will do this, after all I do use Vim, we'll see.

Take your time, "release" apart, no idea when any of this can/will be done. I love it, it is great for what it does.