</head>
<body>
-<h1>Blërg Documentation</h1>
+<h1>Blërg</h1>
+
+Blërg is a minimalistic tagged text document database engine that also
+pretends to be a <a href="/">microblogging system</a>. It is designed
+to efficiently store small (< 64K) pieces of text in a way that they
+can be quickly retrieved by record number or by querying for tags
+embedded in the text. Its native interface is HTTP — Blërg comes
+as either a standalone HTTP server, or a CGI. Blërg is written in pure
+C.
+
+<ul class="toc">
+ <li><a href="#installing">Installing</a>
+ <ul>
+ <li><a href="#getting_the_source">Getting the source</a></li>
+ <li><a href="#requirements">Requirements</a></li>
+ <li><a href="#configuring">Configuring</a></li>
+ <li><a href="#building">Building</a></li>
+ <li><a href="#installing">Installing</a></li>
+ </ul>
+ </li>
+ <li><a href="#design">Design</a>
+ <ul>
+ <li><a href="#motivation">Motivation</a></li>
+ <li><a href="#web_app_stack">Web App Stack</a></li>
+ <li><a href="#database">Database</a></li>
+ <li><a href="#problems_and_future_work">Problems and Future Work</a></li>
+ </ul>
+ </li>
+</ul>
-<h2>Installing</h2>
+<h2><a name="installing">Installing</a></h2>
-<h3>Getting the source</h3>
+<h3><a name="getting_the_source">Getting the source</a></h3>
<p>There's no stable release, yet, but you can get everything currently
running on blerg.dominionofawesome.com by cloning the git repository at
http://git.bytex64.net/blerg.git.
-<h3>Requirements</h3>
+<h3><a name="requirements">Requirements</a></h3>
<p>Blërg has varying requirements depending on how you want to run it
— as a standalone HTTP server, or as a CGI. You will need:
<li><a href="http://www.newbreedsoftware.com/cgi-util/download/">cgi-util</a> >= 2.2.1</li>
</ul>
-<h3>Configuring</h3>
+<h3><a name="configuring">Configuring</a></h3>
<p>I know I'm gonna get shit for not using an autoconf-based system, but
I really didn't want to waste time figuring it out. You should edit
suggestions on how to make Blërg more portable, I'd be happy to hear
them.
-<h3>Building</h3>
+<h3><a name="building">Building</a></h3>
<p>At this point, it should be gravy. Type 'make' and in a few seconds,
you should have <code>http_blerg</code>, <code>cgi_blerg</code>,
individually as well, if you, for example, don't want to install the
prerequisites for <code>http_blerg</code> or <code>cgi_blerg</code>.
-<h3>Installing</h3>
+<h3><a name="installing">Installing</a></h3>
<p>While it's not required, Blërg will be easier to set up if you
configure it to work from the root of your website. For this reason,
it's at /rss.cgi).
-<h2>Design</h2>
+<h2><a name="api">API</a></h2>
+
+<p>Blërg's API was designed to be as simple as possible. Data sent from
+the client is POSTed with the application/x-www-form-urlencoded
+encoding, and a successful response is always JSON. The API endpoints
+will be described as though the server were serving requests from the
+root of the wesite.
+
+<h3><a name="api_definitions">API Definitions</a></h3>
+
+<p>On failure, all API calls return either a standard HTTP error
+response, like 404 Not Found if a record or user doesn't exist, or a 200
+response with some JSON indicating failure, which will look like this:
+
+<p><code>{"status": "failure"}</code>
+
+<p>Blërg doesn't currently explain <i>why</i> there is a failure, and
+I'm not sure it ever will.
+
+<p>On success, you'll either get some JSON relating to your request (for
+/get, /tag, or /info), or a JSON object indicating success (for /create,
+/put, /login, or /logout), which looks like this:
+
+<p><code>{"status": "success"}</code>
+
+<p>For the CGI backend, you may get a 500 error if something goes wrong.
+For the HTTP backend, you'll get nothing (since it will have crashed),
+or maybe a 502 Bad Gateway if you have it behind another web server.
+
+<p>All usernames must be 32 characters or less. Usernames must contain
+only the ASCII characters 0-9, A-Z, a-z, underscore (_), period (.),
+hyphen (-), single quote ('), and space ( ). Passwords can be at most
+64 bytes, and have no limits on characters (but beware: if you have a
+null in the middle, it will stop checking there because I use
+<code>strncmp(3)</code> to compare).
+
+<p>Tags must be 64 characters or less, and can contain only the ASCII
+characters 0-9, A-Z, a-z, hyphen (-), and underscore (_).
+
+<h3><a name="api_create">/create</a> - create a new user</a></h3>
+
+<p>To create a user, POST to /create with <code>username</code> and
+<code>password</code> parameters for the new user. The server will
+respond with failure if the user exists, or if the user can't be created
+for some other reason. The server will respond with success if the user
+is created.
+
+<h3><a name="api_login">/login</a> - log in</a></h3>
-<h3>Motivation</h3>
+<p>POST to /login with the <code>username</code> and
+<code>password</code> parameters for an existing user. The server will
+respond with failure if the user does not exist or if the password is
+incorrect. On success, the server will respond with success, and will
+set a cookie named 'auth' that must be sent by the client when accessing
+restricted API functions (/put and /logout).
+
+<h3><a name="api_logout">/logout</a> - log out</a></h3>
+
+<p>POST to /logout with with <code>username</code>, the user to log out,
+along with the auth cookie in a Cookie header. The server will respond
+with failure if the user does not exist or if the auth cookie is bad.
+The server will respond with success after the user is successfully
+logged out.
+
+<h3><a name="api_put">/put</a> - add a new record</a></h3>
+
+<p>POST to /put with <code>username</code> and <code>data</code>
+parameters, and an auth cookie. The server will respond with failure
+if the auth cookie is bad, if the user doesn't exist, or if
+<code>data</code> contains more than 65535 bytes <i>after</i> URL
+decoding. The server will respond with success after the record is
+successfully added.
+
+<h3><a name="api_get">/get/(user), /get/(user)/(start record)-(end record)</a> - get records for a user</a></h3>
+
+<p>A GET request to /get/(user), where (user) is the user desired, will
+return the last 50 records for that user in a list of objects. The
+record objects look like this:
+
+<pre>
+{
+ "record":"0",
+ "timestamp":1294309438,
+ "data":"eatin a taco on fifth street"
+}
+</pre>
+
+<p><code>record</code> is the record number, <code>timestamp</code> is
+the UNIX epoch timestamp (i.e., the number of seconds since Jan 1 1970
+00:00:00 GMT), and <code>data</code> is the content of the record. The
+record number is sent as a string because while Blërg supports record
+numbers up to 2<sup>64</sup> - 1, Javascript uses floating point for all
+its numbers, and can only support integers without truncation up to
+2<sup>53</sup>. This difference is largely academic, but I didn't want
+this problem to sneak up on anyone who is more insane than I am. :]
+
+<p>The second form, /get/(user)/(start record)-(end record), retrieves a
+specific range of records, from (start record) to (end record)
+inclusive. You can retrieve at most 100 records this way. If (end
+record) - (start record) specifies more than 100 records, the server
+will respond with JSON failure.
+
+<h3><a name="api_info">/info/(user)</a> - Get information about a user</a></h3>
+
+<p>A GET request to /info/(user) will return a JSON object with
+information about the user (currently only the number of records). The
+info object looks like this:
+
+<pre>
+{
+ "record_count": "544"
+}
+</pre>
+
+<p>Again, the record count is sent as a string for 64-bit safety.
+
+<h3><a name="api_tag">/tag/(#|H|@)(tagname)</a> - Retrieve records containing tags</a></h3>
+
+<p>A GET request to this endpoint will return the last 50 records
+associated with the given tag. The first character is either # or H for
+hashtags, or @ for mentions (I call them ref tags). You should URL
+encode the # or @, lest some servers complain at you. The H alias for #
+was created because Apache helpfully strips the fragment of a URL
+(everything from the # to the end) before handing it off to the CGI,
+even if the hash is URL encoded. The record objects also contain an
+extra <code>author</code> field, like so:
+
+{
+ "author":"Jon",
+ "record":"57",
+ "timestamp":1294555793,
+ "data":"I'm taking #garfield to the vet."
+}
+
+<p>There is currently no support for getting more than 50 tags, but /tag
+will probably mutate to work like /get.
+
+<h2><a name="design">Design</a></h2>
+
+<h3><a name="motivation">Motivation</a></h3>
<p>Blërg was created as the result of a thought experiment: "What if
Twitter didn't need thousands of servers? What if its millions of users
could be handled by a single highly efficient server?" This is probably
an unreachable goal due to the sheer amount of I/O, but we could
-certainly do better. Blërg was designed with very simple requirements:
+certainly do better. Blërg was thus designed as a system with very
+simple requirements:
<ol>
<li>Store and fetch small chunks of text efficiently</li>
<p>And to further simplify, I didn't bother handling deletes, full text
search, or more complicated tag searches. Blërg only does the basics.
-<h3>Web App Stack</h3>
+<h3><a name="web_app_stack">Web App Stack</a></h3>
<table class="pizzapie">
<tr><th>Classical model</th></tr>
href="http://couchdb.apache.org/">CouchDB</a>, though I didn't know that
until after I wrote Blërg. :)
-<h3>Database</h3>
+<h3><a name="database">Database</a></h3>
<p>Early in the design process, I decided to blatantly copy <a
href="http://www.varnish-cache.org/">varnish</a> and rely heavily on
searched by opening the tag file, reading the last 50 entries or so, and
then reading all the records listed. Voila, fast tag lookups.
+<p>At this point, you're probably thinking, "Is that it?" Yep, that's
+it. Blërg isn't revolutionary, it's just a system whose requirements
+were pared down until the implementation could be made dead simple.
+
<p>Also, keeping with the style of modern object databases, I haven't
implemented any data safety (har har). Blërg does not sync anything to
disk before returning success. This should make Blërg extremely fast,
and totally unreliable in a crash. But that's the way you want it,
right? :]
-<h3>Problems and Future Work</h3>
+<h3><a name="problems_and_future_work">Problems and Future Work</a></h3>
+
+<p>Blërg probably doesn't actually work like Twitter because I've never
+actually had a Twitter account.
<p>I couldn't find a really good fast HTTP server library.
Libmicrohttpd is small, but it's focused on embedded applications, so it
projects / blerg.git / blobdiff