ntred - controller/hub/client interface to a cluster of btred servers
To query the servers:
ntred [-m|-I macro-file] [-e code] [--hub hub] [--port port]
[-N|-H] [-T] [--key-file file] [--filelist file-list ] [-L files]
-- script-arguments
To start remote servers and a hub:
ntred -i [--servers server[,server,...]] [--serverlist server-list]
[--filelist file-list] [--max-servers num] [--no-secondary-files]
[--no-server-check] [--no-server-start] [--no-hub-start]
[--old-dist-method] [--safe-mode] [--server-debug]
[-m|-I macro-file] [-M module] [--btred path-to-btred]
[--ssh ssh-command] [--local]
[--key-file] [--hub hub] [--port port] [file [...]]
To close all remote servers and a hub:
ntred --quit|--ps-quit
To kill all remote servers and a hub:
ntred --kill|--ps-kill
To manage files on the servers:
ntred --list-files|--list-changed-files
ntred --reload-files
ntred --reload-macros [-m macro-file]
ntred --load-files [--filelist file-list] [file [...]]
ntred --close-files
ntred --save-files|--save-changed-files [-s strip-sfx]
[-a append-sfx] [-p strip-prefix] [-r add-prefix] [-f out-fmt]
ntred --dump-files [--filelist file-list] [file [...]]
ntred --upload-file filename < fs-file
Get help:
ntred -u for usage (synopsis) ntred -h for help ntred --man for the manual page
This program is able to start one or more btred servers on a set
of host machines over SSH, create a proxy hub to provide the
communication between the servers and a client, distribute given
files over the servers (provided the servers are able to load the
files from given filenames (eg. they share the files over NFS),
query the servers using a btred-macro and collect the answers. It
is highly recommended to use some password-free authentication
method for SSH (e.g. Kerberos or ssh-agent), so that
password input is not required each time the SSH connection is made
(at least two per-host).
In the client mode, the standard output of the macro is printed
to STDOUT of the client. STDERR is reserved for debugging and
information messages as well as error messages caused by the macros
on the servers. The rest of the error output from a server is
stored in a file <logdir>/ntred-server-<host>.log
(where <logdir> can be specified using --logdir,
NTRED_LOGDIR or TMP or TEMP environment variable and defaults to
/tmp if none of the previous exists).
ntred [-m macro-file] [-e code] [--hub hub] [--port port]
[-N|-H] [-T] [--key-file file] -- script-arguments
--macro-file or some Perl one-liner. If omitted, it
defaults to 'autostart()' and a macro with this name must be
defined in the macro file provided..--execute.Note that servers already have the set of macros they obtained on startup (i.e. --init): it is namely either the default set of macros (tred.mac) or the set specified via --macro-file, and beside one of these also the set specified via --include-macro-file.
--macro-file and is only provided for compatibility
with btred. If both these options are used, both sets
of macros are loaded.if ($root) { do {{ CODE }} while TredMacro::NextTree()
} loop).while
($this) { CODE ; $this=$this-following }``;> loop).while ($this) { CODE ;
$this=$this-following_visible(FS()) }``;>
loop).--listed-files|-L
but this time the files to be processed are listed in the given
file rather than on the command line. Both options may be used
together in which case the file-lists are joined.
ntred -i [--servers server[,server,...]] [--serverlist server-list]
[--filelist file-list] [--max-servers num]
[--no-secondary-files] [--no-server-check]
[--no-server-start] [--no-hub-start] [--old-dist-method]
[--safe-mode] [--server-debug] [--max-retries num]
[-m macro-file] [-M module] [--btred path-to-btred]
[--ssh ssh-command] [--local] [--key-file file]
[--hub hub] [--port port] [file [...]]
--serverlist.--servers nor --serverlist is
provided, then the list of servers is read from
~/.ntred_serverlist.In the safe mode, only the following opcodes and opcode-sets are allowed (see Opcode):
:base_core :base_mem :base_loop :base_math entereval caller dofile print entertry leavetry tie untie bless sprintf localtime gmtime sort require
plus :base_orig, but the following opcodes (which are forbidden):
getppid getpgrp setpgrp getpriority setpriority pipe_op sselect select dbmopen dbmclose tie untie
--macro-file to load macros from both
filename and the default macro set
from (tred.mac). --macro-file can still be
used in combination with --include-macro-file to
supply a replacement for tred.mac.ntred --list-files|--list-changed-files
ntred --reload-files [-filelist file-list] [--listed-files file [...]]
ntred --reload-changed-files
ntred --reload-macros [-m macro_file]
ntred --load-files [--filelist file-list] [file [...]]
ntred --close-files
ntred --save-files|--save-changed-files [-s strip-sfx]
[-a append-sfx] [-p strip-prefix] [-r add-prefix] [-f out-fmt] [--knit]
ntred --quit
ntred --kill|--ps-kill [--servers server[,server,...]] [--serverlist server-list]
ntred --break|--ps-break
ntred --dump-files [--filelist file-list] [file [...]]
ntred --upload-file filename < fs-file
--break
but tries to identify btred server processes by looking at the
output of the system command ps x -o pid,command. This
may help if killall -USR1 btred doesn't work.killall -9 ntred on the local machine and and
killall -9 btred on the server hosts listed with
--servers, --serverlist, or in ~/.ntred_serverlist.--kill
but tries to identify btred server processes by looking at the
output of the system command ps x -o pid,command. This
may help if killall -TERM btred doesn't work.$TredMacro::FileChanged variable to 1, otherwise the
btred server would never notice.--reload-files
request).--filelist or --listed-files options
are given, reload only files occuring in the given lists (all other
files remain intact in servers' memory).-m (--macro-file) is specified,
the servers use the given macro-file instead of the original one
(specified when initializing btred servers). Note, that the file
(with exactly the same path) must be visible from all server
hosts.--filelist of in
~/.ntred_filelist to the servers. Note, that a file distributed to
a server is not reloaded by the server if the server already has a
file with the same path in memory.--add-prefix, --strip-prefix,
--strip-suffix, --append-suffix.--save-files except
that only files that have been changed by some macro will be saved.
Note, that a macro has to claim that the file was changed by
setting $TredMacro::FileChanged variable to 1,
otherwise the btred server would never notice. See also --list-changed-files.PMLBackend which supports so called
knitting, i.e. a method to pull certain data from
external resources and push it back (with all changes) to the
original position in the resource when saving the file. This option
allows to list the types of resources (in PML the types are the
reference names listed in the PML schema) which should be saved.
Default is NONE. This type of resources doesn't
include so called secondary files.##n suffix where n is the absolute
position of the tree in the file (starting from one). The following
example shows how csplit command can be used to save
individual dumps into separate files:
ntred --dump <files> | csplit -z -f out -b '%d.fs' - '/\/\/FSEND/+2' '{*}'
To merge these separate files into one huge FS file, use
any2any -m hugeout.fs out*.fs
##n suffix where n is the absolute
position of the tree in the file (starting from one).glob function on the filename patterns
given on the command-line. This expands possible wild-card patterns
on each of the filename command-line argument as the standard Unix
shell /bin/csh would do. This can not only help in a situation
where the shell used doesn't support wildcard expansion, but can
also be used to reduce the number of the command-line arguments
passed to the process in cases where the argument list would after
the shell-expansion exceed a system limit. Note, that currently
expansion is performed on the client regardless of the type of
request. This may change in the future versions.USE AT YOUR OWN RISK. IF SECURITY IS A CRITICAL ISSUE OR IF IN DOUBT, DON'T USE IT AT ALL.
Why is security an issue here? Because btred servers execute
almost arbitrary Perl code provided by the client. In the
--unrestricted mode such code may contain arbitrary
commands such as system() or open(). It is therefore
desirable that the servers are not open for all parties.
The following precautions have been taken to lower the potential security risks:
1) Both btred servers and hub require an authorization based on verification of a MD5 signature of a random data block (generated by the server in case of the hub-to-btred-server communication and by the hub in case of the client-to-hub communication) xor-ed with an authorization key known to both parties. Although the communication is unencrypted, the client must with each request send a MD5 checksum of the request XORed with the secret authorization key. Only requests whose signature is verified by the server, are responded to.
2) There may be only one connection from a hub to a server. As soon as it is closed, the server terminates.
3) If the servers are started by the hub itself (using
--init) the authorization key is created by the hub
and is passed to the btred server via a ssh encrypted pipe.
4) For the client's disposal, the authorization key is stored in
user's home directory as ~/.ntred_session_key with
permissions set to 600 (only user can read or write). This
theoretically (depending on the general security of the system)
limits the access to the hub (and thus to the servers) to the user
running the hub only. It may, though, be obviously abused from the
local root account to execute arbitrary perl code on all btred
server hosts. This might especially be undesirable if the hub runs
on a machine whose administrator would normaly have no user access
to the machines running btred servers. Another possible security
issue might arise if user's home directory is on a remote NFS
server, so that NFS intervenes accessing the key file. Since NFS
uses an unencrypted protocol, network sniffing techniques may be
used to obtain the authorization key and hence run arbitrary code
on btred hosts. If such situations are likely to happen (e.g. in a
large network) it is advisable to use a different location for the
authorization key (see --key-file), e.g.
/tmp.
5) It is possible to restrict Perl code evaluated on the servers
to a safer compartment, where some critical Perl commands are
disabled. In some cases, these restrictions may not be sufficient,
in other they may be too strict. Some memory leaks can appear when
Safe compartment is used. See --safe-mode above for
more discussion.
6) Unless --hub option is used, the hub runs on
localhost and as such is not (under normal circumstances) open for
connections from the outside world. If you are considering making
the hub listen on a non-local interface, note that it is a much
better option to configure a secure SSH tunnel.
~/.ntred_serverlist - default list of servers to
use
~/.ntred_filelist - default list of files to load
on servers
~/.ntred_session_key - client/hub session key
Petr Pajas <pajas@matfyz.cz>
Zdenek Zabokrtsky <zabokrtsky@ufal.mff.cuni.cz>
Copyright 2003-2005 Petr Pajas and Zdenek Zabokrtsky, All rights reserved.