The ‘py’ lib provides a uniform high-level api to deal with filesystems
and filesystem-like interfaces:
py.path. It aims to offer a central
object to fs-like object trees (reading from and writing to files, adding
files/directories, examining the types and structure, etc.), and out-of-the-box
provides a number of implementations of this API.
py.path.local - local file system path¶
basic interactive example¶
The first and most obvious of the implementations is a wrapper around a local filesystem. It’s just a bit nicer in usage than the regular Python APIs, and of course all the functionality is bundled together rather than spread over a number of modules.
>>> import py >>> temppath = py.path.local('py.path_documentation') >>> foopath = temppath.join('foo') # get child 'foo' (lazily) >>> foopath.check() # check if child 'foo' exists False >>> foopath.write('bar') # write some data to it >>> foopath.check() True >>> foopath.read() 'bar' >>> foofile = foopath.open() # return a 'real' file object >>> foofile.read(1) 'b'
object oriented interface to os.path and other local filesystem related information.
raised on pyimport() if there is a mismatch of __file__’s
Exception.with_traceback(tb) – set self.__traceback__ to tb and return self.
return True if ‘other’ references the same file as ‘self’.
remove a file or directory (or a directory tree if rec=1). if ignore_errors is True, errors while removing directories will be ignored.
return hexdigest of hashvalue for this file.
create a modified version of this path. the following keyword arguments modify various path parts:
a:/some/path/to/a/file.ext xx drive xxxxxxxxxxxxxxxxx dirname xxxxxxxx basename xxxx purebasename xxx ext
return the directory path joined with any given path arguments.
return a new path by appending all ‘args’ as path components. if abs=1 is used restart from root if any of the args is an absolute path.
open(mode='r', ensure=False, encoding=None)¶
return an opened file with the given mode.
If ensure is True, create parent directories if needed.
check a path for existence and properties.
Without arguments, return True if the path exists, otherwise False.
file=1 # is a file file=0 # is not a file (may not even exist) dir=1 # is a dir link=1 # is a link exists=1 # exists
You can specify multiple checker definitions, for example:
path.check(file=1, link=1) # a link pointing to a file
list directory contents, possibly filter by the given fil func and possibly sorted.
return size of the underlying file object
return last modification time of the path.
copy(target, mode=False, stat=False)¶
copy path to target.
If mode is True, will copy copy permission from path to target. If stat is True, copy permission, last modification time, last access time, and flags from path to target.
rename this path to target.
pickle object into path location
create & return the directory joined with args.
write binary data into path. If ensure is True create missing parent directories.
write_text(data, encoding, ensure=False)¶
write text data into path using the specified encoding. If ensure is True create missing parent directories.
write(data, mode='w', ensure=False)¶
write data into path. If ensure is True create missing parent directories.
ensure that an args-joined path exists (by default as a file). if you specify a keyword argument ‘dir=True’ then the path is forced to be a directory path.
Return an os.stat() tuple.
Return an os.lstat() tuple.
set modification time for the given path. if ‘mtime’ is None (the default) then the file’s mtime is set to current time.
Note that the resolution for ‘mtime’ is platform dependent.
change directory to self and return old current directory
Return a context manager, which changes to the path’s dir during the managed “with” context. On __enter__ it returns the old dir, which might be
return a new path which contains no symbolic links.
return last access time of the path.
change permissions to the given mode. If mode is an integer it directly encodes the os-specific modes. if rec is True perform recursively.
return the Python package path by looking for the last directory upwards which still contains an __init__.py. Return None if a pkgpath can not be determined.
return path as an imported python module.
If modname is None, look for the containing package and construct an according module name. The module will be put/looked up in sys.modules. if ensuresyspath is True then the root dir for importing the file (taking __init__.py files into account) will be prepended to sys.path if it isn’t there already. If ensuresyspath==”append” the root dir will be appended if it isn’t already contained in sys.path. if ensuresyspath is False no modification of syspath happens.
Special value of ensuresyspath==”importlib” is intended purely for using in pytest, it is capable only of importing separate .py files outside packages, e.g. for test suite without any __init__.py file. It effectively allows having same-named test modules in different places and offers mild opt-in via this option. Note that it works only in recent versions of python.
return stdout text from executing a system child process, where the ‘self’ path points to executable. The process is directly invoked and not through a system shell.
sysfind(name, checker=None, paths=None)¶
return a path object found by looking at the systems underlying PATH specification. If the checker is not None it will be invoked to filter matching paths. If a binary cannot be found, None is returned Note: This is probably not working on plain win32 systems but may work on cygwin.
return the system’s temporary directory (where tempfiles are usually created in)
return a Path object pointing to a fresh new temporary directory (which we created ourself).
basename part of path.
return a string which is a relative path from self (assumed to be a directory) to dest such that self.join(bestrelpath) == dest and if not such path can be determined return dest.
chown(user, group, rec=0)¶
change ownership to the given user and group. user and group may be specified by a number or by a name. if rec is True change ownership recursively.
return the common part shared with the other path or None if there is no common part.
dirname part of path.
ensure the path joined with args is a directory.
extension of the path (including the ‘.’).
return true if the basename/fullname matches the glob-‘pattern’.
valid pattern characters:
* matches everything ? matches any single character [seq] matches any character in seq [!seq] matches any char not in seq
If the pattern contains a path-separator then the full path is used for pattern matching and a ‘*’ is prepended to the pattern.
if the pattern doesn’t contain a path-separator the pattern is only matched against the basename.
(deprecated) return object unpickled from self.read()
make_numbered_dir(prefix='session-', rootdir=None, keep=3, lock_timeout=172800)¶
return unique directory with a number greater than the current maximum one. The number is assumed to start directly after prefix. if keep is true directories with a number less than (maxnum-keep) will be removed. If .lock files are used (lock_timeout non-zero), algorithm is multi-process safe.
posix style hard link to another name.
create a symbolic link with the given value (pointing to another name).
move this path to target.
return a root-first list of all ancestor directories plus the path itself.
pure base name of the path.
read and return a bytestring from reading the path.
read and return a bytestring from reading the path.
read and return a Unicode string from reading the path.
read and return a list of lines from the path. if cr is False, the newline will be removed from the end of each line.
return value of a symbolic link.
return a string which is the relative part of the path to the given ‘relpath’.
visit(fil=None, rec=None, ignore=<class 'py._path.common.NeverRaised'>, bf=False, sort=False)¶
yields all paths below the current one
fil is a filter (glob pattern or callable), if not matching the path will not be yielded, defaulting to None (everything is returned)
rec is a filter (glob pattern or callable) that controls whether a node is descended, defaulting to None
ignore is an Exception class that is ignoredwhen calling dirlist() on any of the paths (by default, all exceptions are reported)
bf if True will cause a breadthfirst search instead of the default depthfirst. Default: False
sort if True will sort entries within each directory level.
py.path implementations that the py lib provides wrap the
popular Subversion revision control system: the first (called ‘svnurl’)
by interfacing with a remote server, the second by wrapping a local checkout.
Both allow you to access relatively advanced features such as metadata and
versioning, and both in a way more user-friendly manner than existing other
Some example usage of
.. >>> import py .. >>> if not py.test.config.option.urlcheck: raise ValueError('skipchunk') >>> url = py.path.svnurl('http://codespeak.net/svn/py') >>> info = url.info() >>> info.kind 'dir' >>> firstentry = url.log()[-1] >>> import time >>> time.strftime('%Y-%m-%d', time.gmtime(firstentry.date)) '2004-10-02'
Example usage of
.. >>> if not py.test.config.option.urlcheck: raise ValueError('skipchunk') >>> temp = py.path.local('py.path_documentation') >>> wc = py.path.svnwc(temp.join('svnwc')) >>> wc.checkout('http://codespeak.net/svn/py/dist/py/path/local') >>> wc.join('local.py').check() True
Common vs. specific API, Examples¶
All Path objects support a common set of operations, suitable for many use cases and allowing to transparently switch the path object within an application (e.g. from “local” to “svnwc”). The common set includes functions such as path.read() to read all data from a file, path.write() to write data, path.listdir() to get a list of directory entries, path.check() to check if a node exists and is of a particular type, path.join() to get to a (grand)child, path.visit() to recursively walk through a node’s children, etc. Only things that are not common on ‘normal’ filesystems (yet), such as handling metadata (e.g. the Subversion “properties”) require using specific APIs.
A quick ‘cookbook’ of small examples that will be useful ‘in real life’, which also presents parts of the ‘common’ API, and shows some non-common methods:
Searching .txt files¶
Search for a particular string inside all files with a .txt extension in a specific directory.
>>> dirpath = temppath.ensure('testdir', dir=True) >>> dirpath.join('textfile1.txt').write('foo bar baz') >>> dirpath.join('textfile2.txt').write('frob bar spam eggs') >>> subdir = dirpath.ensure('subdir', dir=True) >>> subdir.join('textfile1.txt').write('foo baz') >>> subdir.join('textfile2.txt').write('spam eggs spam foo bar spam') >>> results =  >>> for fpath in dirpath.visit('*.txt'): ... if 'bar' in fpath.read(): ... results.append(fpath.basename) >>> results.sort() >>> results ['textfile1.txt', 'textfile2.txt', 'textfile2.txt']
Working with Paths¶
This example shows the
py.path features to deal with
filesystem paths Note that the filesystem is never touched,
all operations are performed on a string level (so the paths
don’t have to exist, either):
>>> p1 = py.path.local('/foo/bar') >>> p2 = p1.join('baz/qux') >>> p2 == py.path.local('/foo/bar/baz/qux') True >>> sep = py.path.local.sep >>> p2.relto(p1).replace(sep, '/') # os-specific path sep in the string 'baz/qux' >>> p2.bestrelpath(p1).replace(sep, '/') '../..' >>> p2.join(p2.bestrelpath(p1)) == p1 True >>> p3 = p1 / 'baz/qux' # the / operator allows joining, too >>> p2 == p3 True >>> p4 = p1 + ".py" >>> p4.basename == "bar.py" True >>> p4.ext == ".py" True >>> p4.purebasename == "bar" True
This should be possible on every implementation of
regardless of whether the implementation wraps a UNIX filesystem, a Windows
one, or a database or object tree, these functions should be available (each
with their own notion of path seperators and dealing with conversions, etc.).
Checking path types¶
Now we will show a bit about the powerful ‘check()’ method on paths, which allows you to check whether a file exists, what type it is, etc.:
>>> file1 = temppath.join('file1') >>> file1.check() # does it exist? False >>> file1 = file1.ensure(file=True) # 'touch' the file >>> file1.check() True >>> file1.check(dir=True) # is it a dir? False >>> file1.check(file=True) # or a file? True >>> file1.check(ext='.txt') # check the extension False >>> textfile = temppath.ensure('text.txt', file=True) >>> textfile.check(ext='.txt') True >>> file1.check(basename='file1') # we can use all the path's properties here True
As an example of ‘uncommon’ methods, we’ll show how to read and write
properties in an
.. >>> if not py.test.config.option.urlcheck: raise ValueError('skipchunk') >>> wc.propget('foo') '' >>> wc.propset('foo', 'bar') >>> wc.propget('foo') 'bar' >>> len(wc.status().prop_modified) # our own props 1 >>> msg = wc.revert() # roll back our changes >>> len(wc.status().prop_modified) 0
Some uncommon functionality can also be provided as extensions, such as SVN authentication:
.. >>> if not py.test.config.option.urlcheck: raise ValueError('skipchunk') >>> auth = py.path.SvnAuth('anonymous', 'user', cache_auth=False, ... interactive=False) >>> wc.auth = auth >>> wc.update() # this should work >>> path = wc.ensure('thisshouldnotexist.txt') >>> try: ... path.commit('testing') ... except py.process.cmdexec.Error, e: ... pass >>> 'authorization failed' in str(e) True
Known problems / limitations¶
- The SVN path objects require the “svn” command line, there is currently no support for python bindings. Parsing the svn output can lead to problems, particularly regarding if you have a non-english “locales” setting.
- While the path objects basically work on windows, there is no attention yet on making unicode paths work or deal with the famous “8.3” filename issues.