David Howells [Tue, 13 May 2014 16:23:37 +0000 (17:23 +0100)]
rxgen: Handle bulk arguments correctly
Need to check for a bulk-class argument of an integer-type before testing for
the integer-type otherwise the integer-type is selected. This causes the wrong
argument spec to be emitted for PyArg_ParseTuple() when decoding the arguments
for an RPC call to be issued.
Signed-off-by: David Howells <dhowells@redhat.com>
David Howells [Thu, 1 May 2014 14:53:03 +0000 (15:53 +0100)]
Implement "bos getlog" and reengineer the decode path somewhat
Implement the "bos getlog" subcommand - which requires making the receive side
of split RPC function work. To this end, the following reengineering has been
done to the decode side:
(1) Lift the requirement that the Rx buffer cursor be 4-byte aligned as
BOZO.GetLog() doesn't align the log data it returns.
(2) Permit a request for an unknown - and possibly zero - amount of data to be
set in call->need_size. BOZO.GetLog() doesn't tell you in advance how
much data it will return: the data stops when the Rx receive phase ends
with LAST_PACKET being flagged.
(3) Separate blob decoding (strings & opaques) from bulk decoding in the
member variables of the rx_call struct, where a bulk decode is now a
sequence of blobs, structs or ints.
(4) Handle blob decoding asynchronously, where the buffer into which a blob is
being written may not represent contiguous memory. This is done by
setting up with an init function, called once, and a decode-into function,
called repeatedly whilst it returns 1. The function is only called when
there is sufficient data in the receive buffers.
(5) Decode padding asynchronously by working out up front for a blob how much
padding it requires and then decoding it as its own blob at the end using
a special source buffer as a marker to switch processing.
(6) Perform split reception by adding additional states within the decode
state machine to call out to the handler functions.
The receive() method of the split_handler class provided to a split RPC
function then implements its own state machine on top of the decoder state
machine.
Further:
(*) rxrpc_recv_data() now reports ENODATA rather than EMSGSIZE if the receive
phase ends with short data.
(*) MSG_MORE handling has separate rx_call struct members for the send and
receive phase now to avoid confusion.
(*) rxrpc_enc/dec_slow() now use rxrpc_enc/dec_blob() to avoid duplicating a
lot of code.
(*) rxrpc_enc/dec() now also go to the slow path if the cursor is misaligned,
which may mean the data to be read is split across buffers.
(*) The rx_call struct now has a pointer to a cleanup function to clean up the
decoder state at the end to deal with aborted calls.
(*) rxrpc_post_dec() reduces call->need_size as well as call->data_count as we
don't want to be waiting for N bytes to turn up for a size-N blob if we
have already received N-1 bytes of it and only need one more byte.
This does mean that call->need_size may need resetting more often.
(*) Added an output_raw() python output method to permit bytearray objects to
be printed. Conversion to a string means that control characters and
quotes get escaped (eg. a newline char gets converted to \n).
(*) Catching the SystemExit exception should not produce an error due to ret
not existing as a variable.
Signed-off-by: David Howells <dhowells@redhat.com>
David Howells [Tue, 15 Apr 2014 08:34:42 +0000 (09:34 +0100)]
rxgen: Implement the transmission side of split-mode RPC calls
Implement the transmission side of RPC calls that are marked with a 'split'
flag - ie. functions in which the caller is allowed to interpolate an
arbitrary transmission blob after the arguments and to receive an arbitrary
blob before the response is received.
Signed-off-by: David Howells <dhowells@redhat.com>
David Howells [Wed, 16 Apr 2014 07:18:07 +0000 (08:18 +0100)]
rxgen: Use PyErr_NoMemory() rather than PyExc_MemoryError
Use PyErr_NoMemory() rather than PyExc_MemoryError as the former actually
raises an exception. The latter is just a class and is then just an object
with no special meaning when returned.
Signed-off-by: David Howells <dhowells@redhat.com>
David Howells [Tue, 15 Apr 2014 09:42:23 +0000 (10:42 +0100)]
rxgen: Fix rxrpc_send_data()
Make three Fixes to rxrpc_send_data():
(1) Advance to the next buffer when filling the iovec array rather than
endlessly cycling over the first one.
(2) Only clear MSG_MORE if we have sent all the buffered data at the end of
the transmission phase of a call.
(3) Don't turn the return value of sendmsg() into just -1 or 0 as we need the
number of bytes actually sent to allow us to correctly advance the buffer
cursor.
Signed-off-by: David Howells <dhowells@redhat.com>
David Howells [Mon, 14 Apr 2014 12:42:41 +0000 (13:42 +0100)]
Fix up output and error displaying in commands
Fix up output and error displaying in commands so far implemented. Replace
usage of print() with output() and outputf() for normal operation and error()
for error display. debug() is replaced with verbose() which is contingent on
the -verbose switch.
Also add error handling to a number of commands.
Signed-off-by: David Howells <dhowells@redhat.com>
David Howells [Mon, 14 Apr 2014 07:09:58 +0000 (08:09 +0100)]
rxgen: The abort table comparator needs to handle big numbers
The abort table comparator needs to handle big numbers (larger than
0x7fffffff). Subtracting a small number from such a big number and then
casting to a signed int results in a negative number, which bsearch()
interprets as indicating that the bigger number is smaller than the smaller
number.
Signed-off-by: David Howells <dhowells@redhat.com>
David Howells [Sat, 12 Apr 2014 14:51:03 +0000 (15:51 +0100)]
Provide separate arg interpretation functions for multivalued arguments
Provide separate arg interpretation functions for multivalued arguments so
that the single-valued versions return a non-list and the multivalued returns
a list.
Signed-off-by: David Howells <dhowells@redhat.com>
David Howells [Sat, 12 Apr 2014 13:33:46 +0000 (14:33 +0100)]
Use gethostbyname_ex() and gethostbyaddr() rather than the DNS resolver
Use gethostbyname_ex() and gethostbyaddr() for A and PTR record retrieval
rather than using the DNS resolver directly so that /etc/hosts and suchlike
are also used.
Signed-off-by: David Howells <dhowells@redhat.com>
David Howells [Fri, 11 Apr 2014 22:57:53 +0000 (23:57 +0100)]
rxgen: The string and bytes decode functions must advance the cursor
The string and bytes decode functions (py_dec_string() and rxrpc_dec_bytes())
need to advance the extraction cursor after extracting so the data doesn't get
reused.
Signed-off-by: David Howells <dhowells@redhat.com>
David Howells [Thu, 10 Apr 2014 13:04:52 +0000 (14:04 +0100)]
Implement "vos examine"
Implement the "vos examine" command. There is a lot of common display code
between this and several of the other commands, so the common bits are drawn
out into a library module for all to share.
Signed-off-by: David Howells <dhowells@redhat.com>
David Howells [Thu, 10 Apr 2014 10:19:56 +0000 (11:19 +0100)]
Save raw argument list for each switch
Several of the switch parameters have their argument lists cooked by the
argument parser. This means we cannot get at the original string(s) when
producing error messages.
Save the raw argument list for each switch in the parameters dictionary as a
list of strings attached to a key of "raw.<switchname>".
Signed-off-by: David Howells <dhowells@redhat.com>
David Howells [Thu, 10 Apr 2014 10:17:25 +0000 (11:17 +0100)]
Add a convenience module for partition ID handling
Add a convenience module for partition ID handling. One function translates
"/vicepX[Y]", "vicepX[Y]", "X[Y]" and "N" into a numeric ID and range checks
it. Another function translates a numeric ID back into "/vicepX[Y]".
Make the argument parser use it.
Signed-off-by: David Howells <dhowells@redhat.com>
David Howells [Wed, 9 Apr 2014 22:38:32 +0000 (23:38 +0100)]
rxgen: Emit individual exception classes for defined abort codes and raise them
Emit an exception class for each defined abort code and raise the appropriate
exception when an abort is received. Each set of classes is subclassed from a
package-specific class (eg. AbortVL_NOENT -> VLAbort) and each
package-specific class is subclassed from a generic class (eg. VLAbort ->
RemoteAbort) so that all package codes can be trapped in one statement and all
remote aborts can be trapped together.
If an unknown code is received, then this is raised as RemoteAbort only.
RxRPC/rxgen codes are supplied now also in the rxgen.xg file under the RXGEN
package name.
KAA/KAM/KAT codes are under the KA package.
Signed-off-by: David Howells <dhowells@redhat.com>
David Howells [Wed, 9 Apr 2014 20:27:58 +0000 (21:27 +0100)]
rxgen: Set struct members correctly in struct types
When setting a member of a struct (eg. VL.ListAddrByAttributes) where that
member is itself a struct type (eg. afsUUID), we need to check that the type
of the value to assign is the member type, not the container type.
Signed-off-by: David Howells <dhowells@redhat.com>
David Howells [Wed, 9 Apr 2014 14:40:36 +0000 (15:40 +0100)]
Move VL connection security determination into cell.py
Move the Volume Location DB connection security handling into
cell.open_vl_server() as it's common to all vos commands. It can be
configured directly by passing in a dictionary holding the appropriate
parameters.
Signed-off-by: David Howells <dhowells@redhat.com>
David Howells [Wed, 9 Apr 2014 14:43:22 +0000 (15:43 +0100)]
Provide generic check for non-combinable parameters
Provide generic check for non-combinable parameters in command py files. A
list of tuples of pairs of argument names that don't interact is provided by
the name cant_combine_arguments as a global var.
Signed-off-by: David Howells <dhowells@redhat.com>
David Howells [Tue, 8 Apr 2014 17:23:54 +0000 (18:23 +0100)]
Allow RxRPC transport security to be requested
Honour the -encrypt flag to vos listvldb and make it tell AF_RXRPC to secure
the connection. This requires an appropriate key to be available in one of
the user's keyrings, labelled:
afs@<UPPERCASE.CELL.NAME>
Without such a key, error:
[Errno 126] Required key not available
will be issued.
Signed-off-by: David Howells <dhowells@redhat.com>
David Howells [Wed, 8 Jan 2014 23:38:07 +0000 (23:38 +0000)]
Initial code commit now that the VLDB lookup test python script works
The rxgen.pl script takes a C header file, parses it for #defined constants,
structures declarations and synchronous RPC call function declarations and
emits the following:
- C functions to marshal/unmarshal structs.
- C functions to marshal func arguments and dispatch the request.
- C functions to unmarshal the replies.
- C functions to provide synchronous RPC calls matching the source.
- A python module containing:
- Python types to wrap the structs.
- Python methods to wrap the synchronous RPC calls.
- Python constants to wrap the constants.
It cannot yet handle functions that deal in bulk types/lists of structs and
functions that take interpolated chunks of arbitrary data.
It also does not yet emit server-side parts or top layer of the asynchronous
client side stuff.
It requires python v3 and requires the AF_RXRPC socket family to be available
in the kernel.
A script called "vl-test.py" is included that can be used to test the
functionality. Run without arguments, it will attempt to retrieve the
"afs.root" volume from the grand.central.org AFS cell.
The test script uses the DNS so may require a python3-dns package installing.
Signed-off-by: David Howells <dhowells@redhat.com>
projects / users / dhowells / kafs-utils.git / log