I've built something along these lines, Object Shell: http://geophile.com/osh, (GPL license). Object Shell provides for piping of python objects between commands. For example,
osh invokes the interpreter. ls is osh's ls command. The -f flag restricts the ls command to files. ^ is the osh pipe character, so the list of files in the current directory is piped to the next command. select filters out objects that don't pass the predicate, in this case, that file.size > 20000. $ renders output (using python's str()).
osh also does database access, e.g.
osh sql "select * from my_table" $
and remote access. E.g., to run the same query on every node of a cluster named foobar:
osh @foobar [ sql "select count(*) from my_table" ] $
Piping python tuples between commands and then operating on them from the command line is really handy.
Object Shell also has a python API, e.g.
#!/usr/bin/python
from osh.api import *
osh(remote('foobar',
sql("select count(*) from my_table")),
out())
that looks really nice. Is there an interactive version, like ipython but with pipelining? Also is there any way to use pipe for pipelining rather than carat (a bit sad but Im used to reading pipe as pipe after twenty years of Unix and DOS).
No, there isn't an interactive version, but I once tried the API from inside iPython, and that worked.
Pipe is interpreted by the shell and separates the commands on either side, so I don't see how that could work. The very first version of osh actually did run separate commands, connected by a Unix pipe, but the pickling and unpickling costs were extremely high, so I opted for running the whole command in one process.
osh also does database access, e.g.
and remote access. E.g., to run the same query on every node of a cluster named foobar: Piping python tuples between commands and then operating on them from the command line is really handy.Object Shell also has a python API, e.g.