Object Citation Numbering - ocn
(a text positioning system)
Search - "granular"
Of interest is the ease of streaming documents to a relational database, at an object (roughly paragraph) level and the potential for increased precision in the presentation of matches that results thereby. The ability to serialise html, LaTeX, XML, SQL, (whatever) is also inherent in / incidental to the design. For a description see the abandoned U.S. provisional patent application
License, SiSU is licensed under GPLv3 or later
Gnu / Linux / Unix
Note: the placement of SiSU documents on the Net predate the release of SiSU.
SiSU Short Description
SiSU is a comprehensive future-proofing electronic document management system. Built-in search capabilities allow you to search across multiple documents and highlight matches in an easy-to-follow format. Paragraph numbering system allows you to cite your electronic documents in a consistent manner across multiple file formats. Multiple format outputs allow you to display your documents in plain text, PDF (portrait and horizontal), OpenDocument format, HTML, or e-book reading format (EPUB). Word mapping allows you to easily create word indexes for your documents. Future-proofing flexibility allows you to quickly adapt your documents to newer output formats as needed. All these and many other features are achieved with little or no additional work on your documents - by marking up the documents with a super simplistic markup language, leaving the SiSU engine to handle the heavy-lifting processing.
Potential users of SiSU include individual authors who want to publish their books or articles electronically to reach a broad audience, web publishers who want to provide multiple channels of access to their electronic documents, or any organizations which centrally manage a medium or large set of electronic documents, especially governmental organizations which may prefer to keep their documents in easily accessible yet non-proprietary formats.
SiSU is an Open Source project initiated and led by Ralph Amissah <email@example.com> and can be contacted via mailing list <http://lists.sisudoc.org/listinfo/sisu> at <firstname.lastname@example.org>. SiSU is licensed under the GNU General Public License.
For less markup than the most elementary HTML you can have more.
SiSU - Structured information, Serialized Units for electronic documents, is an information structuring, transforming, publishing and search framework with the following features:
(i) markup syntax: (a) simpler than html, (b) mnemonic, influenced by mail/messaging/wiki markup practices, (c) human readable, and easily writable,
(ii) (a) minimal markup requirement, (b) single file marked up for multiple outputs,
* documents are prepared in a single UTF-8 file using a minimalistic mnemonic syntax. Typical literature, documents like "War and Peace" require almost no markup, and most of the headers are optional.
* markup is easily readable/parsed by the human eye, (basic markup is simpler and more sparse than the most basic html), [this may also be converted to XML representations of the same input/source document].
* markup defines document structure (this may be done once in a header pattern-match description, or for heading levels individually); basic text attributes (bold, italics, underscore, strike-through etc.) as required; and semantic information related to the document (header information, extended beyond the Dublin core and easily further extended as required); the headers may also contain processing instructions.
(iii) (a) multiple outputs primarily industry established and institutionally accepted open standard formats, include amongst others: plaintext (UTF-8); html; (structured) XML; ODF (Open Document text); EPUB; LaTeX; PDF (via LaTeX); SQL type databases (currently PostgreSQL and SQLite). Also produces: concordance files; document content certificates (md5 or sha256 digests of headings, paragraphs, images etc.) and html manifests (and sitemaps of content). (b) takes advantage of the strengths implicit in these very different output types, (e.g. PDFs produced using typesetting of LaTeX, databases populated with documents at an individual object/paragraph level, making possible granular search (and related possibilities))
(iv) outputs share a common numbering system (dubbed "object citation numbering" (ocn)) that is meaningful (to man and machine) across various digital outputs whether paper, screen, or database oriented, (PDF, html, XML, EPUB, sqlite, postgresql), this numbering system can be used to reference content.
(v) SQL databases are populated at an object level (roughly headings, paragraphs, verse, tables) and become searchable with that degree of granularity, the output information provides the object/paragraph numbers which are relevant across all generated outputs; it is also possible to look at just the matching paragraphs of the documents in the database; [output indexing also work well with search indexing tools like hyperesteier].
(vi) use of semantic meta-tags in headers permit the addition of semantic information on documents, (the available fields are easily extended)
(vii) creates organised directory/file structure for (file-system) output, easily mapped with its clearly defined structure, with all text objects numbered, you know in advance where in each document output type, a bit of text will be found (e.g. from an SQL search, you know where to go to find the prepared html output or PDF etc.)... there is more; easy directory management and document associations, the document preparation (sub-)directory may be used to determine output (sub-)directory, the skin used, and the SQL database used,
(viii) "Concordance file" wordmap, consisting of all the words in a document and their (text/ object) locations within the text, (and the possibility of adding vocabularies),
(ix) document content certification and comparison considerations: (a) the document and each object within it stamped with an md5 hash making it possible to easily check or guarantee that the substantive content of a document is unchanged, (b) version control, documents integrated with time based source control system, default RCS or CVS with use of $Id$ tag, which SiSU checks
(x) SiSU's minimalist markup makes for meaningful "diffing" of the substantive content of markup-files,
(xi) easily skinnable, document appearance on a project/site wide, directory wide, or document instance level easily controlled/changed,
(xii) in many cases a regular expression may be used (once in the document header) to define all or part of a documents structure obviating or reducing the need to provide structural markup within the document,
(xiii) prepared files may be batch process, documents produced are static files so this needs to be done only once but may be repeated for various reasons as desired (updated content, addition of new output formats, updated technology document presentations/representations)
(xiv) possible to pre-process, which permits: the easy creation of standard form documents, and templates/term-sheets, or; building of composite documents (master documents) from other sisu marked up documents, or marked up parts, i.e. import documents or parts of text into a main document should this be desired
there is a considerable degree of future-proofing, output representations are "upgradeable", and new document formats may be added.
(xv) there is a considerable degree of future-proofing, output representations are "upgradeable", and new document formats may be added: (a) modular, (thanks in no small part to Ruby) another output format required, write another module.... (b) easy to update output formats (eg html, XHTML, LaTeX/PDF produced can be updated in program and run against whole document set), (c) easy to add, modify, or have alternative syntax rules for input, should you need to,
(xvi) scalability, dependent on your file-system (ext3, Reiserfs, XFS, whatever) and on the relational database used (currently Postgresql and SQLite), and your hardware,
(xvii) only marked up files need be backed up, to secure the larger document set produced,
(xviii) document management,
(xix) Syntax highlighting for SiSU markup is available for a number of text editors.
(xx) remote operations: (a) run SiSU on a remote server, (having prepared sisu markup documents locally or on that server, i.e. this solution where sisu is installed on the remote server, would work whatever type of machine you chose to prepare your markup documents on), (b) generated document outputs may be posted by sisu to remote sites (using rsync/scp) (c) document source (plaintext utf-8) if shared on the net may be identified by its url and processed locally to produce the different document outputs.
(xxi) document source may be bundled together (automatically) with associated documents (multiple language versions or master document with inclusions) and images and sent as a zip file called a sisupod, if shared on the net these too may be processed locally to produce the desired document outputs, these may be downloaded, shared as email attachments, or processed by running sisu against them, either using a url or the filename.
(xxii) for basic document generation, the only software dependency is Ruby, and a few standard Unix tools (this covers plaintext, html, XML, ODF, EPUB, LaTeX). To use a database you of course need that, and to convert the LaTeX generated to PDF, a LaTeX processor like tetex or texlive.
as a developers tool it is flexible and extensible
SiSU ("SiSU information Structuring Universe" or "Structured information, Serialized Units"),1 is a Unix command line oriented framework for document structuring, publishing and search. Featuring minimalistic markup, multiple standard outputs, a common citation system, and granular search.
Using markup applied to a document, SiSU can produce plain text, HTML, XHTML, XML, OpenDocument, LaTeX or PDF files, and populate an SQL database with objects2 (equating generally to paragraph-sized chunks) so searches may be performed and matches returned with that degree of granularity (e.g. your search criteria is met by these documents and at these locations within each document). Document output formats share a common object numbering system for locating content. This is particularly suitable for "published" works (finalized texts as opposed to works that are frequently changed or updated) for which it provides a fixed means of reference of content.
How it works
SiSU markup is fairly minimalistic, it consists of: a (largely optional) document header, made up of information about the document (such as when it was published, who authored it, and granting what rights) and any processing instructions; and markup within text which is related to document structure and typeface. SiSU must be able to discern the structure of a document, (text headings and their levels in relation to each other), either from information provided in the instruction header or from markup within the text (or from a combination of both). Processing is done against an abstraction of the document comprising of information on the document's structure and its objects,2 which the program serializes (providing the object numbers) and which are assigned hash sum values based on their content. This abstraction of information about document structure, objects, (and hash sums), provides considerable flexibility in representing documents different ways and for different purposes (e.g. search, document layout, publishing, content certification, concordance etc.), and makes it possible to take advantage of some of the strengths of established ways of representing documents, (or indeed to create new ones).
1. also chosen for the meaning of the Finnish term "sisu".
2 objects include: headings, paragraphs, verse, tables, images, but not footnotes/endnotes which are numbered separately and tied to the object from which they are referenced.
SiSU was developed in relation to legal documents, and is strong across a wide variety of texts (law, literature...(humanities, law and part of the social sciences)). SiSU handles images but is not suitable for formulae/ statistics, or for technical writing at this time.
SiSU has been developed and has been in use for several years. Requirements to cover a wide range of documents within its use domain have been explored.
w3 since October 3 1993