<head>
<title>Blërg Documentation</title>
<link rel="stylesheet" href="/css/doc.css">
+<meta http-equiv="content-type" content="text/html; charset=utf8">
</head>
<body>
C.
<ul class="toc">
- <li><a href="#installing">Installing</a>
+ <li><a href="#running">Running Blërg</a>
<ul>
<li><a href="#getting_the_source">Getting the source</a></li>
<li><a href="#requirements">Requirements</a></li>
<li><a href="#api">API</a>
<ul>
<li><a href="#api_definitions">API Definitions</a></li>
+ <li><a href="#api_authorization">Authorizaton</a></li>
<li><a href="#api_create">/create - create a new user</a></li>
<li><a href="#api_login">/login - log in</a></li>
<li><a href="#api_logout">/logout - log out</a></li>
<li><a href="#api_info">/info/(user) - Get information about a user</a></li>
<li><a href="#api_tag">/tag/(#|H|@)(tagname) - Retrieve records containing tags</a></li>
<li><a href="#api_subscribe">/subscribe/(user) - Subscribe to a user's updates</a></li>
- <li><a href="#api_unsubscribe">/unsubscribe/(user) - Unsubscribe from a user's updates</a></li>
<li><a href="#api_feed">/feed - Get updates for subscribed users</a></li>
- <li><a href="#api_feedinfo">/feedinfo, /feedinfo/(user) - Get subscription status</a></li>
+ <li><a href="#api_status">/status, /status/(user) - Get or clear general and user-specific status</a></li>
<li><a href="#api_passwd">/passwd - Change a user's password</a></li>
</ul>
</li>
+ <li><a href="#libraries">Libraries</a>
+ <ul>
+ <li><a href="#lib_c">C</a></li>
+ <li><a href="#lib_perl">Perl</a></li>
+ </ul>
+ </li>
<li><a href="#design">Design</a>
<ul>
<li><a href="#motivation">Motivation</a></li>
</li>
</ul>
-<h2><a name="installing">Installing</a></h2>
+<h2><a name="running">Running Blërg</a></h2>
<h3><a name="getting_the_source">Getting the source</a></h3>
— as a standalone HTTP server, or as a CGI. You will need:
<ul>
-<li><a href="http://lloyd.github.com/yajl/">yajl</a> >= 1.0.0
+<li><a href="http://lloyd.github.com/yajl/">yajl</a> >= 1.0.0 and < 2
(yajl is a JSON parser/generator written in C which, by some twisted
sense of humor, requires ruby to compile)</li>
</ul>
<h3><a name="configuring">Configuring</a></h3>
-<p>There is now an experimental autoconf build system. If you run
-<code>add-autoconf</code>, it'll do the magic and create a
-<code>configure</code> script that'll do the familiar things. If I ever
-get around to distributing source packages, you should find that this
-has already been done.
-
-<p>If you'd rather stick with the manual system, you should edit libs.mk
-and put in the paths where you can find headers and libraries for the
-above requirements.
+<p>Edit libs.mk and put in the paths where you can find headers and
+libraries for the above requirements.
<p>Also, further apologies to BSD folks — I've probably committed
several unconscious Linux-isms. It would not surprise me if the
easier than yoursite.com/blerg/). If you do want to put it in a
subdirectory, you will have to modify <code>www/js/blerg.js</code> and
change baseURL at the top as well as a number of other self-references
-in that file and <code>www/index.html</code>. The CGI version should
-work fine this way, but the HTTP version will require the request to be
-rewritten, as it expects to be serving from the root.
+in that file and <code>www/index.html</code>.
<p>You cannot serve the database and client from different domains
(i.e., yoursite.com vs othersite.net, or even foo.yoursite.com and
bar.yoursite.com). This is a requirement of the web browser — the
same origin policy will not allow an AJAX request to travel across
-domains.
+domains (though you can probably get around it these days with <a
+ href="http://en.wikipedia.org/wiki/Cross-origin_resource_sharing">Cross-origin
+ resource sharing</a>).
-<h4>For the standalone web server:</h4>
-
-<p>Right now, <code>blerg.httpd</code> doesn't serve any static assets,
-so you're going to have to put it behind a real webserver like apache,
-lighttpd, nginx, or similar. Set the document root to the www
-directory, then proxy /info, /create, /login, /logout, /get, /tag, and
-/put to blerg.httpd. You can change the port <code>blerg.httpd</code>
-listens on in <code>config.h</code>.
-
-<h4>For the CGI version:</h4>
+<h4>For straight CGI with Apache</h4>
<p>Copy the files in www/ to the root of your web server. Copy
<code>blerg.cgi</code> to your web server. Included in www-configs/ is
call the CGI something other than <code>blerg.cgi</code>, the .htaccess
file will need to be modified.
+<h4>For nginx</h4>
+
+<p>Nginx can't run CGI directly, and there's currently no FastCGI
+version of Blërg, so you will have to run it under some kind of CGI to
+FastCGI gateway, like the one described <a
+href="http://wiki.nginx.org/SimpleCGI">here on the nginx wiki</a>. This
+pretty much destroys the performance of Blërg, but it's all we've got
+right now.
+
<h4>The extra RSS CGI</h4>
<p>There is an optional RSS cgi (<code>rss.cgi</code>) that will serve
RSS feeds for users. Install this like <code>blerg.cgi</code> above.
+As of 1.9.0, this is a perl FastCGI script, so you will have to make
+sure the perl libraries are available to it. A good way of doing that
+is to install to an environment directory, as described below.
+
+<h4>Installing to an environment directory</h4>
+
+<p>The Makefile has support for installing Blërg into a directory that
+includes tools, libraries, and configuration snippets for shell and web
+servers. Use it as <code>make install-environment
+ ENV_DIR=<directory></code>. Under <directory>/etc will be
+a shell script that sets environment variables, and configuration
+snippets for nginx and apache to do the same. This should make it
+somewhat easier to use Blërg in a self-contained way.
+
+<p>For example, this will install Blërg to an environment directory
+inside your home directory:
+
+<pre>user@devhost:~/blerg$ make install-environment ENV_DIR=$HOME/blerg-env
+...
+user@devhost:~/blerg$ . ~/blerg-env/etc/env.sh
+</pre>
+
+<p>Then, you will be able to run tools like <code>blergtool</code>, and
+it will operate on data inside <code>~/blerg-env/data</code>. Likewise,
+you can include
+<code>/home/user/blerg-env/etc/nginx-fastcgi-vars.conf</code> or
+<code>/home/user/blerg-env/etc/apache-setenv.conf</code> in your
+webserver to make the CGI/FastCGI scripts to the same thing.
<h2><a name="api">API</a></h2>
response, like 404 Not Found if a record or user doesn't exist, or a 200
response with a 'JSON failure', which will look like this:
-<p><code>{"status": "failure"}</code>
+<pre>{"status": "failure"}</pre>
<p>Blërg doesn't currently explain <i>why</i> there is a failure, and
I'm not sure it ever will.
/get, /tag, or /info), or a 'JSON success' response (for /create, /put,
/login, or /logout), which looks like this:
-<p><code>{"status": "success"}</code>
+<pre>{"status": "success"}</pre>
<p>For the CGI backend, you may get a 500 error if something goes wrong.
For the HTTP backend, you'll get nothing (since it will have crashed),
<p>Tags must be 64 characters or less, and can contain only the ASCII
characters 0-9, A-Z, a-z, underscore (_), and hyphen (-).
+<h3><a name="api_authorization">Authorization</a></h3>
+
+<p>As the result of a successful <a href="#api_login">login</a>, the server
+will send back a cookie named <code>auth</code>. This cookie authorizes
+restricted requests, and must be sent for any API endpoint marked <span
+class="feature">authorization</span>, or else you will get a 403 Forbidden
+response. The cookie format looks like:
+
+auth=username/abcdef0123456789abcdef0123456789
+
+That is a username, a forward slash, and 32 hexadecimal digits which denote the
+"token" identifying the session. On logout, the server will invalidate the
+token and expire the cookie.
+
<h3><a name="api_create">/create</a> - create a new user</a></h3>
<p>To create a user, POST to /create with <code>username</code> and
respond with JSON failure if the user does not exist or if the password
is incorrect. On success, the server will respond with JSON success,
and will set a cookie named 'auth' that must be sent by the client when
-accessing restricted API functions (/put and /logout).
+accessing restricted API functions (See <a
+href="#api_authorization">Authorization</a> above).
<h3><a name="api_logout">/logout</a> - log out</a></h3>
+<div class="feature">authorization</div>
-<p>POST to /logout with with <code>username</code>, the user to log out,
-along with the auth cookie in a Cookie header. The server will respond
-with JSON failure if the user does not exist or if the auth cookie is
-bad. The server will respond with JSON success after the user is
-successfully logged out.
+<p>POST to /logout. The server will respond with JSON failure if the
+user does not exist or if the request is unauthorized. The server will
+respond with JSON success after the user is successfully logged out.
<h3><a name="api_put">/put</a> - add a new record</a></h3>
+<div class="feature">authorization</div>
-<p>POST to /put with <code>username</code> and <code>data</code>
-parameters, and an auth cookie. The server will respond with JSON
-failure if the auth cookie is bad, if the user doesn't exist, or if
-<code>data</code> contains more than 65535 bytes <i>after</i> URL
-decoding. The server will respond with JSON success after the record is
-successfully added.
+<p>POST to /put with a <code>data</code> parameter. The server will
+respond with JSON failure if the request is unauthorized, if the user
+doesn't exist, or if <code>data</code> contains more than 65535 bytes
+<i>after</i> URL decoding. The server will respond with JSON success
+after the record is successfully added.
<h3><a name="api_get">/get/(user), /get/(user)/(start record)-(end record)</a> - get records for a user</a></h3>
<h3><a name="api_subscribe">/subscribe/(user)</a> - Subscribe to a
user's updates</a></h3>
+<div class="feature">authorization</div>
-<p>POST to /subscribe/(user) with a <code>username</code> parameter and
-an auth cookie, where (user) is the user whose updates you wish to
-subscribe to. The server will respond with JSON failure if the auth
-cookie is bad or if the user doesn't exist. The server will respond
-with JSON success after the subscription is successfully registered.
-
-<h3><a name="api_unsubscribe">/unsubscribe/(user)</a> - Unsubscribe from
-a user's updates</h3>
-
-<p>Identical to /subscribe, but removes the subscription.
+<p>POST to /subscribe/(user) with a <code>subscribed</code> parameter
+that is either "true" or "false", indicating whether (user) should be
+subscribed to or not. The server will respond with JSON failure if the
+request is unauthorized or if the user doesn't exist. The server will
+respond with JSON success after the subscription request is successfully
+registered.
<h3><a name="api_feed">/feed</a> - Get updates for subscribed users</h3>
+<div class="feature">authorization</div>
<p>POST to /feed, with a <code>username</code> parameter and an auth
cookie. The server will respond with a JSON list of the last 50 updates
from all subscribed users, in reverse chronological order. Fetching
-/feed resets the new message count returned from /feedinfo.
+/feed does not reset the new message count returned from /status. To do
+that, look at <a href="#api_status">POST /status</a>.
<p>NOTE: subscription notifications are only stored while subscriptions
are active. Any records inserted before or after a subscription is
active will not show up in /feed.
-<h3><a name="api_feedinfo">/feedinfo, /feedinfo/(user)</a> - Get subscription
-status for a user</a></h3>
+<h3><a name="api_status">/status, /status/(user)</a> - Get or clear
+general and user-specific status</a></h3>
+<div class="feature">authorization</div>
-<p>POST to /feedinfo with a <code>username</code> parameter and an auth
-cookie to get general information about your subscribed feeds.
-Currently, this only tells you how many new records there are since the
-last time /feed was fetched. The server will respond with a JSON
-object:
+<p>GET to /status to get information about your account. It tells you
+the number of new subscription records since the last time the
+subscription counter was reset, and a flag for whether the account was
+mentioned since the last time the mention flag was cleared. The server
+will respond with a JSON object:
<pre>
-{"new":3}
+{
+ "feed_new": 3,
+ "mentioned": false
+}
</pre>
-<p>POST to /feedinfo/(user) with a <code>username</code> parameter and
-an auth cookie, where (user) is a user whose subscription status you are
-interested in. The server will respond with a simple JSON object:
+<p>POST to /status with a <code>clear</code> parameter that is either
+"feed" or "mentioned" to reset either the subscription counter or the
+mention flag, respectively. There is not currently a way to clear both
+with a single request. The server will respond with JSON success.
+
+<p>GET to /status/(user) to get subscription information for a
+particular user. The server will respond with a simple JSON object:
<pre>
{"subscribed":true}
the subscription status.
<h3><a name="api_passwd">/passwd</a> - Change a user's password</a></h3>
+<div class="feature">authorization</div>
-<p>POST to /passwd with a <code>username</code> parameter and an auth
-cookie, plus <code>password</code> and <code>new_password</code>
-parameters to change the user's password. For extra protection,
-changing a password requires sending the user's current password in the
-<code>password</code> parameter. If authentication is successful and
-the password matches, the user's password is set to
-<code>new_password</code> and the server responds with JSON success.
+<p>POST to /passwd with <code>password</code> and
+<code>new_password</code> parameters to change the user's password. For
+extra protection, changing a password requires sending the user's
+current password in the <code>password</code> parameter. If
+authentication is successful and the password matches, the user's
+password is set to <code>new_password</code> and the server responds
+with JSON success.
If the password doesn't match, or one of <code>password</code> or
<code>new_password</code> are missing, the server returns JSON failure.
+<h2><a name="libraries">Libraries</a></h2>
+
+<h3><a name="lib_c">C</a></h3>
+
+<p>Most of Blërg's core functionality is packaged in a static library
+called <code>blerg.a</code>. It's not designed to be public or
+installed with `make install-environment`, but it should be relatively
+straightforward to use it in C programs. Look at the headers under the
+<code>database</code> directory.
+
+<p>A secondary library called <code>blerg_auth.a</code> handles the
+authentication layer of Blërg. To use it, look at
+<code>common/auth.h</code>.
+
+<h3><a name="lib_perl">Perl</a></h3>
+
+<p>As of 1.9.0, Blërg includes a perl library called
+<code>Blerg::Database</code>. It wraps the core and authentication
+functionality in a perlish interface. The module has its own POD
+documentation, which you can read with your favorite POD reader, from
+the manual installed in an environment directory, or in HTML <a
+href="perl/Blerg-Database.html">here</a>.
+
<h2><a name="design">Design</a></h2>
<h3><a name="motivation">Motivation</a></h3>
</table>
<p>Blërg does both by smashing the last two or three layers into one
-application. Blërg can be run as either a standalone web server, or as
-a CGI (FastCGI support is planned, but I just don't care right now).
-Less waste, more throughput. As a consequence of this, the entirety of
-the application logic that the user sees is implemented in the client
-app in Javascript. That's why all the URLs have #'s — the page is
-loaded once and switched on the fly to show different views, further
-reducing load on the server. Even parsing hash tags and URLs are done
-in client JS.
+application. Blërg can be run as either a standalone web server
+(currently deprecated because maintaining two versions is hard), or as a
+CGI (FastCGI support is planned, but I just don't care right now). Less
+waste, more throughput. As a consequence of this, the entirety of the
+application logic that the user sees is implemented in the client app in
+Javascript. That's why all the URLs have #'s — the page is loaded
+once and switched on the fly to show different views, further reducing
+load on the server. Even parsing hash tags and URLs are done in client
+JS.
<p>The API is simple and pragmatic. It's not entirely RESTful, but is
rather designed to work well with web-based front-ends. Client data is
Blërg has their own database, which consists of a metdata file, and one
or more data and index files. The data and index files are memory
mapped, which hopefully makes things more efficient by letting the OS
-handle when to read from disk (or maybe not &mdash I haven't benchmarked
-it). The index files are preallocated because I believe it's more
-efficient than writing to it 40 bytes at a time as records are added.
-The database's limits are reasonable:
+handle when to read from disk (or maybe not — I haven't
+benchmarked it). The index files are preallocated because I believe
+it's more efficient than writing to it 40 bytes at a time as records are
+added. The database's limits are reasonable:
<table class="statistics">
<tr><td>maximum record size</td><td>65535 bytes</td></tr>
-<tr><td>maximum number of records per database</td><td>2<sup>64</sup> - 1 bytes</td></tr>
+<tr><td>maximum number of records per database</td><td>2<sup>64</sup> - 1</td></tr>
<tr><td>maximum number of tags per record</td><td>1024</td></tr>
<table>
<p>So as not to create grossly huge and unwieldy data files, the
database layer splits data and index files into many "segments"
-containing at most 64K entries each. Those of you doing some quick math
-in your heads may note that this could cause a problem on 32-bit
-machines — if a full segment contains entries of the maximum
-length, you'll have to mmap 4GB (32-bit Linux gives each process only
-3GB of virtual address space). Right now, 32-bit users should change
+containing at most 64K entries each. Those of you doing some quick
+mental math may note that this could cause a problem on 32-bit machines
+— if a full segment contains entries of the maximum length, you'll
+have to mmap 4GB (32-bit Linux gives each process only 3GB of virtual
+address space). Right now, 32-bit users should change
<code>RECORDS_PER_SEGMENT</code> in <code>config.h</code> to something
lower like 32768. In the future, I might do something smart like not
mmaping the whole fracking file.
projects / blerg.git / blobdiff