README (5032B)
1 pubsubhubbubblub 2 ---------------- 3 4 Generic pubsubhubbub client implementation. 5 It has some helper scripts to use it with RSS/Atom feeds and with sfeed. 6 7 8 What is it 9 ---------- 10 11 pubsubhubbub is a publisher/subscriber technology used to push updates in a 12 webhook-like way. 13 This allows to push content updates, instead of polling for data in an interval. 14 15 16 Features 17 -------- 18 19 - Relatively simple. 20 - Uses pledge(2) and unveil(2) on OpenBSD. 21 - Signature verification support (hub.secret), Pubsubhub 0.4 core SHA1 only. 22 23 24 Dependencies 25 ------------ 26 27 - C compiler (C99). 28 - libc (recommended: C99 and POSIX >= 200809). 29 - httpd and CGI handler. 30 31 32 Optional dependencies 33 --------------------- 34 35 - POSIX make(1) for the Makefile. 36 - POSIX sh(1), used by pubsub_setup. 37 - curl(1) binary: https://curl.haxx.se/ , 38 used by pubsub_setup for subscribing and unsubscribing to a hub. 39 40 41 Files 42 ----- 43 44 pubsub_cgi.c - Small stupid PubSubHubBub implementation as a CGI program. 45 pubsub_gethub - Helper program extract a hub and feed URL from a RSS or Atom feed data. 46 pubsub_setup - Helper script that sets up the directory structure for 47 processing the feed for the CGI program. It has an -s option to 48 subscribe and an -u option to unsubscribe at a hub also. 49 50 51 How to install 52 -------------- 53 54 To build it run: 55 56 make 57 58 Install: 59 60 cp pubsub_gethub pubsub_setup /usr/local/bin 61 cp pubsub_cgi /var/www/cgi-bin/pubsub 62 63 64 A configuration example for the CGI program using OpenBSD httpd and slowcgi, httpd.conf: 65 66 location "/pubsub/**" { 67 request strip 1 68 root "/cgi-bin/pubsub" 69 fastcgi socket "/run/slowcgi.sock" 70 } 71 72 Compile pubsub_cgi.c statically and copy it to /var/www/cgi-bin/pubsub 73 74 - Create a directory with write-access for the pubsub CGI program, for example 75 /var/www/pubsub-data/feedname. The pubsub_setup script can be used to create 76 the directories for a feed. 77 - Make sure to set the proper permissions for the CGI program (slowcgi) and 78 HTTPd, for example www:daemon. 79 - The relative path name of the path to the CGI script (served via the HTTPd) 80 can be changed in the pubsub_setup script. 81 82 83 How does it work 84 ---------------- 85 86 - The CGI program runs at an entrypoint served by a httpd, for example: 87 https://codemadness.org/pubsub/slashdot/secrettoken 88 - This URL is registered at a hub as the callback URL. 89 - When subscribing the hub sends a confirmed request to this callback URL. The CGI program confirms this. 90 - The hub sends then messages using HTTP POST to this callback URL. 91 - The CGI program processes these messages and writes them to a feed directory. 92 - It writes the request entry of a successfuly processed message to a global log file. 93 94 95 Directory structure, relative to for example /var/www/pubsub-data: 96 97 config/feedname/ - Directory with metadata about the feed. 98 config/feedname/callback - The hub.callback URL used to subscribe. 99 config/feedname/hub - The hub URL, for example http://pubsubhubbub.appspot.com/ . 100 config/feedname/topic - hub.topic, the feed URL. 101 config/feedname/secret - hub.secret for calculating the message digest, 102 see Section 8 of Pubsubhubbub core 0.4 (optional). 103 config/feedname/token - File containing a line with a secret token. This makes 104 sure an entrypoint is not easy guessable by different 105 hubs etc (optional). 106 feeds/feedname/ - Directory containing processed messages. 107 tmp/feedname/ - Temporary directory to process messages. 108 Moves to the feeds/feedname directory on success. 109 log - Log file, TAB-separated. 110 111 112 Example 113 ------- 114 115 Get the hub and feed URL: 116 117 curl -s http://rss.slashdot.org/Slashdot/slashdot | pubsub_gethub 118 119 http://rss.slashdot.org/Slashdot/slashdot self 120 http://pubsubhubbub.appspot.com/ hub 121 122 123 Setup the feed structure for the CGI program and also subscribe to the hub using the -s option. 124 125 cd /var/www/pubsub-data 126 pubsub_setup \ 127 -b 'https://codemadness.org/pubsub/' \ 128 -f 'slashdot' \ 129 -h 'http://pubsubhubbub.appspot.com/' \ 130 -t 'http://rss.slashdot.org/Slashdot/slashdot' 131 pubsub_setup -s -f 'slashdot' 132 133 134 Unsubscribe from a hub: 135 136 cd /var/www/pubsub-data 137 pubsub_setup -u -f 'slashdot' 138 139 140 Monitor script example 141 ---------------------- 142 143 This monitors the log file using tail(1) -f for new messages. It uses sfeed 144 and sfeed_plain to process the new message (assuming it's RSS/Atom) and formats 145 it to a plain-text list. 146 147 #!/bin/sh 148 cd /var/www/pubsub-data 149 tail -f log | \ 150 LC_ALL=C awk '{ print $2 "\t" $3; fflush(); }' | \ 151 while IFS=" " read -r feed file; do sfeed < "feeds/${feed}/${file}"; done | \ 152 sfeed_plain 153 154 155 This can then be piped to the suckless ii(1) program for IRC notifications. 156 Of course the sfeed_plain program can be replaced by sfeed_mbox and a MTA for 157 mail notifications too. 158 159 160 References 161 ---------- 162 163 Pubsubhubbub core 0.4: https://pubsubhubbub.github.io/PubSubHubbub/pubsubhubbub-core-0.4.html