sabRE - an alternative multi-user web frontend

[sander]

I have no access to my Ubuntu laptop right now, so I tried on my Ubuntu 12.04.4 VPS, which resulted in the error below. It has node v0.6.12. That seems to be the cause: I solved the error by adding the PPA for node.js, which gave node.js version v0.10.29:

Code: Select all

sudo add-apt-repository ppa:chris-lea/node.js

Then the "npm install" and "run.sh" ran OK. Setup was easy and worked. Good!
I'm now downloading a NZB, and will post back the results.

Code: Select all

npm install
npm http GET https://registry.npmjs.org/coffee-script
npm http GET https://registry.npmjs.org/express
npm http GET https://registry.npmjs.org/jade
npm http GET https://registry.npmjs.org/underscore
npm http GET https://registry.npmjs.org/body-parser
npm http GET https://registry.npmjs.org/cookie-parser
npm http GET https://registry.npmjs.org/method-override
npm http GET https://registry.npmjs.org/async
npm http GET https://registry.npmjs.org/winston
npm http GET https://registry.npmjs.org/mime
npm http GET https://registry.npmjs.org/exec
npm http GET https://registry.npmjs.org/underscore.string
npm http GET https://registry.npmjs.org/node-ini

npm ERR! Error: failed to fetch from registry: underscore
npm ERR!     at /usr/share/npm/lib/utils/npm-registry-client/get.js:139:12
npm ERR!     at cb (/usr/share/npm/lib/utils/npm-registry-client/request.js:31:9)
npm ERR!     at Request._callback (/usr/share/npm/lib/utils/npm-registry-client/request.js:136:18)
npm ERR!     at Request.callback (/usr/lib/nodejs/request/main.js:119:22)
npm ERR!     at Request.<anonymous> (/usr/lib/nodejs/request/main.js:212:58)
npm ERR!     at Request.emit (events.js:88:20)
npm ERR!     at ClientRequest.<anonymous> (/usr/lib/nodejs/request/main.js:412:12)
npm ERR!     at ClientRequest.emit (events.js:67:17)
npm ERR!     at HTTPParser.onIncoming (http.js:1261:11)
npm ERR!     at HTTPParser.onHeadersComplete (http.js:102:31)
npm ERR! You may report this log at:
npm ERR!     <http://bugs.debian.org/npm>
npm ERR! or use
npm ERR!     reportbug --attach /home/sander/git/sabRE/npm-debug.log npm
npm ERR!
npm ERR! System Linux 3.2.0-63-generic-pae
npm ERR! command "node" "/usr/bin/npm" "install"
npm ERR! cwd /home/sander/git/sabRE
npm ERR! node -v v0.6.12
npm ERR! npm -v 1.1.4
npm ERR! message failed to fetch from registry: underscore
npm ERR!
npm ERR! Additional logging details can be found in:
npm ERR!     /home/sander/git/sabRE/npm-debug.log
npm not ok

[sander]

Download results: Download works, the result is in the sabRE's GUI, but user1's download just arrives in :~/Downloads/complete/. That means (AFAIK) there is no way to share it to a specific user.
I would have expected a directory per user defines in sabRE's config files, and thus for example ~/Downloads/complete/user1/ or <basedir>/user1/. Is that possible?

[sander]

Can you explain what these questions / values mean? Is it how to talk to SAB?

Code: Select all

please configure "remoteAuthHost" with current value "127.0.0.1".
description: "host where remote authentication is running."
new value (or enter to use current): 

please configure "remoteAuthPort" with current value "80".
description: "port where remote authentication is listening on."
new value (or enter to use current): 

please configure "remoteAuthPath" with current value "/remoteauth/".
description: "path for the url to the remote authentication."
new value (or enter to use current):

[realgeizt]

Nice to hear that most stuff works.

Download results: Download works, the result is in the sabRE's GUI, but user1's download just arrives in :~/Downloads/complete/. That means (AFAIK) there is no way to share it to a specific user.
I would have expected a directory per user defines in sabRE's config files, and thus for example ~/Downloads/complete/user1/ or <basedir>/user1/. Is that possible?

Did you create categories for each user in SABnzbd? You have to do that by hand because when the category with the user's name is not defined, SABnzbd ignores it and puts downloads into the regular complete dir. Also, there is no way to control categories using the API it seems.

Can you explain what these questions / values mean? Is it how to talk to SAB?

They are needed when you don't use users.json. sabRE will open the URL that results from host/port/path passing username and password as GET parameters in the path. If the remote URL answers "ok", the user is logged into sabRE, if it is not "ok", login is denied.

[sander]

Ah, in SABnzbd I created category 'user1' with relative directory 'user1', and ... it works. Impressive! Well done!

FWIW the relevant call:

Code: Select all

2014-06-23 19:39:42,235::DEBUG::[interface:421] API-call from 127.0.0.1 [??] {'nzbname': 'blabla.2014.DVDRiP.XviD-ROV', 'pp': '3', 'apikey': '46074d6412486', 'name': '/tmp/blabla.2014.DVDRiP.XviD-ROV.nzb', 'cat': 'user1', 'mode': 'addlocalfile'}
2014-06-23 19:44:23,429::INFO::[postproc:281] Starting PostProcessing on blabla.2014.DVDRiP.XviD-ROV => Repair:True, Unpack:True, Delete:True, Script:None, Cat:user1

[realgeizt]

Nice to hear, thank you!

[sander]

FYI / FWIW This is the setup I did, that results in a correctly working sabRE. The remoteauth part is - despite your explanation - still dark matter to me.

Code: Select all

sander@flappie:~/git/sabRE$ ./run.sh 
info: starting up...
--------------------------------------------------------------------------------
-----------------------------   welcome to sabRE   -----------------------------
--------------------------------------------------------------------------------

it seems this is the first start of the application because no configuration
file could be found.

do you want to run the configuration wizard now? (Y/n) 

it seems that SABnzbd is configured on this system.

reading configuration from sabnzbd.ini...
  set "sabPort" to "8080"
  set "sabApiKey" to "b463b3755ad289e737373319ab6eacf"
reading configuration from SABnzbd using it's api...
  set "scriptDir" to "/home/sander/sabnzbd/post-processing-scripts/"
  set "downloadDir" to "/home/sander/Downloads/complete/"

mandatory settings have been configured.

do you want to run the wizard now to configure other settings? (Y/n) 

please configure "port" with current value "3000".
description: "the port sabRE listens on."
new value (or enter to use current): 

please configure "useCurl" with current value "false".
description: "check urls with curl before enqueueing to SABnzbd?"
new value (or enter to use current): 

please configure "hideOtherUsersData" with current value "false".
description: "when enabled, users can only see stuff they enqueued. files from 
              other users are hidden."
new value (or enter to use current): true

please configure "noPostProcess" with current value "false".
description: "when this is enabled no postprocessing (creation of tar archive) 
              will take place. instead sabRE tries to set the category of the 
              download to the currently logged in user's name so you can set up 
              shares on the server to let users download the files directly. 
              when enabling this, don't forget to configure categories in 
              SABnzbd, one for each user. otherwise files will be downloaded to 
              the regular download dir without placing them in separate 
              directories for each user. enabled postprocessing requires sabRE 
              to run on the same machine as SABnzbd."
new value (or enter to use current): true

please configure "nzbUploadDir" with current value "/tmp/".
description: "directory where uploaded .nzb files will be stored. files will be 
              deleted after they have been enqueued."
new value (or enter to use current): 

please configure "remoteAuthEnabled" with current value "false".
description: "make use of user authentication using a remote url?"
new value (or enter to use current): 

please configure "remoteAuthHost" with current value "127.0.0.1".
description: "host where remote authentication is running."
new value (or enter to use current): 

please configure "remoteAuthPort" with current value "80".
description: "port where remote authentication is listening on."
new value (or enter to use current): 

please configure "remoteAuthPath" with current value "/remoteauth/".
description: "path for the url to the remote authentication."
new value (or enter to use current): 

setup complete. run sabRE again to start it up, then login with a user defined 
in users.json.
sander@flappie:~/git/sabRE$ 

[sander]

BTW: why the pussycat in the upper right corner? Is it like this: sabRE = sabre tiger = cat = pussycat?

[sander]

It's 2014, so the obvious question is: how can I enable sabRE to listen on IPv6?

Now it's only IPv4:

Code: Select all

$ sudo netstat -apon | grep 3000 | grep -i listen
tcp        0      0 0.0.0.0:3000            0.0.0.0:*               LISTEN      7169/nodejs      off (0.00/0/0)

I checked data/settings.json, but it only has '"port": 3000,', not a IPv6 settings. And I tried to read the source code, found "app.listen(settings.port,", but not how to set the IP address to listen to (so: 0.0.0.0 or :: or maybe 127.0.01)

[realgeizt]

Hi sander,

sorry for my late reply!

The remoteauth part is - despite your explanation - still dark matter to me.

I will explain it in detail now. I hope it is understandable. When a user logs in, sabRE checks the users.json file if the user is allowed to login. If the user is not defined in users.json and remoteAuth is enable, sabRE will try to authenticate the user using the remote service. The configuration to do so could look like this:

Code: Select all

  "remoteAuthEnabled": true,
  "remoteAuthHost": "127.0.0.1",
  "remoteAuthPort": 80,
  "remoteAuthPath": "/remoteauth/",

So when the user is not found in users.json, sabRE will make an HTTP GET request to the URL, in this case this would be "http://127.0.0.1/remoteauth/?username=u ... ed-by-user". When the server processing the request by sabRE replies with "ok" as response, the user will be allowed to login into sabRE. When the server does not reply with "ok", login will be denied. You only need this for a special setup, for example when you don't want to define users in users.json and already have another website with user accounts. In my case I already have a website with a forum where me and many friends already have accounts, so I can use this database and do not have to copy every user into users.json.

BTW: why the pussycat in the upper right corner? Is it like this: sabRE = sabre tiger = cat = pussycat?

There is no real reason for this .. I just wanted a nice picture and cats are always good. You can of course change the image on your installation

how can I enable sabRE to listen on IPv6?

Some days ago I searched for it but I did not find out how to solve the problem. Also I have to admit that I am still using IPv4 everywhere... Hopefully I will take another look at this soon.

[realgeizt]

Replace the line "app.listen settings.port, () =>" in app.coffee with "app.listen settings.port, '::', () =>". Then sabRE should listen on IPv6, too.

[sander]

Replace the line "app.listen settings.port, () =>" in app.coffee with "app.listen settings.port, '::', () =>". Then sabRE should listen on IPv6, too.

I changed git/sabRE/cs_app/app.coffee to contain:

Code: Select all

if settings.loaded
  # start the server
  app.listen settings.port, '::', () =>

and now sabRE listens on:

http://[::1]:3000/
http://ip6-localhost:3000/
and on my public IPv6 address. And also on the IPv4 addresses.

Great!

PS: proof:

Code: Select all

$ sudo netstat -apon | grep 3000
tcp6       0      0 :::3000                 :::*                    LISTEN      4249/nodejs      off (0.00/0/0)

[realgeizt]

Very nice. I will push this to GitHub soon.
BTW, if you update your copy after my push, you will have to do "npm install" before starting sabRE. I hope everything works afterwards because I changed some things, if not I am here for new bug reports ...

[realgeizt]

I just pushed a new version. It now listens on IPv6, additionally there is a new option to rename windows executables that can be switched on/off in "sabre_settings.py". I hope everything went right - this was my first commit using Git on Linux from the shell... Don't forget to run "npm install" before launching sabRE after the update.

[realgeizt]

Some minutes ago a new version was published. The postprocessing scripts can now convert .flac files to .mp3 files when the needed tools are installed and the user desired the conversion. Also a new setting to hide downloads after a configurable number of days was introduced. Some other minor stuff changed, but it's not worth to talk about it...

Have a nice weekend!