I'm writing a fuse filesystem using python that should interact with amazon S3 (basically treat S3 buckets as filesystems) and facing some issues with my readdir implementation.
First of all I'd like to mention I'm relatively new to python and fuse (more of the java guy, using python here because of the simplicity of the fuse binding..), so this may all just be a stupid beginner's mistake..
Here's what I currently have:
def readdir(self, path, fh):
s3Path = S3fsUtils.toS3Path(path) # removes prefixed slash - boto3 can't handle that in key names
print("Reading dir: " + str(path))
retVal = [".", ".."]
for s3Obj in self.bucket.objects.all(): # for now list all objects in bucket
tmp = str(s3Obj.key)
if tmp.startswith(s3Path): # only return things below current path
print("READDIR: appending to output: " + tmp)
return retVal # return directory contents as a list of strings
Here's what happens when running "ls -l" (The filesystem is mounted into "/tmp/fusetest"):
... and here's the console output of the filesystem:
(found entries are a few "directories", i.e. S3 keys with no data behind them)
Reading dir: /
READDIR: appending to output: blabla/
READDIR: appending to output: blablubb/
READDIR: appending to output: haha/
READDIR: appending to output: hahaha/
READDIR: appending to output: huhu/
READDIR: appending to output: new_folder/
Releasing dir: /
I'm guessing the problem is that I return a list of strings rather than some more "C-struct-like" thing...
I found this question which is also about problems with readdir, there a class "fuse.Direntry" is used. However, in my fuse.py (fusepy version = 2.0.2), I cannot find any class like that, the closest in name I found was "fuse_file_info" which doesn't really look useful for the task at hand.
So what should readdir return and where does the i/o error come from?
Ok, this turned out to be just what I expected - a stupid mistake...
Since Amazon S3 represents folders as empty files whose names end with a slash, my file listings contained a lot of entries with slashes at the end.
It turned out, fuse cannot handle that, causing the readdir operation to fail.
Removing the slashes from the file names does the trick.