I suppose we could use this mechanism, which we already use for ioctl, for several other file attributes as well, e.g. timestamps. We could put it all under a directory named .attrs-(filename).
As for performance, the filesystem drivers could implement attribute access as separate files while still using attrs/xattrs as the underlying storage mechanism. The main issue with this is when attributes are copied before the files. However, if we make the attributes transparent in libc, this shouldn't be much of an issue, especially if we guarantee that the attributes come after the files in the dirent listing. Overall, I think this could be a relatively clean way to implement both regular and extended attributes. Any other thoughts on this?
On Tue, Apr 2, 2019, 7:54 AM Valery V. Sedletski via users < users@lists.genode.org> wrote:
Norman Feske wrote:
Hello Valery,
thanks for the background info about extended attributes.
mandatory to be existed when opening the file). So, EA's on FAT32 are laying around near the file itself, in the same directory, in a hidden file. The disadvantage is that EA files are laying around everywhere, like a trash. So, if a file system doesn't support EA's, they can be emulated like this.
I admittedly have not much clue about file systems. To me as a user, features like this (this also goes for the strange "resource forks" of old Mac OS) look like glorified directories. They seem to be just more complicated, less transparent, and cause friction when moving data across file systems. Given my poor background, please take the following with a grain of salt.
Yes, also WinNT goes even further: it supports files having multiple streams of data. Resource fork from MacOS or EA can be implemented as streams too. Only Resource fork stores multiple metainfo in one stream, and EA's can be implemented both ways: as a single stream, or multiple streams.
I wonder, in situations where meta data has to contain a lot of domain knowledge, wouldn't sqlite be a good way to organize it?
Probably yes, though, sqlite files are just tables with fixed number of fields. Storing EA in SQLite or any other relational database should require a data schema with multiple tables, which may be too complex, because a file can have any number of EA's, whereas a table contains a fixed number of fields. So, it may require a complicated data schema.
In situations where meta data are simple annotations to files (like storing the icon for a file), why can't both the file and the meta data go to into a directory? This is the way Risc OS (and also today's Mac OS, AFAIK) deal with the situation.
They could be not simple annotations/text data, but they also could be any small binary data attached to a file (icons being a good example).
Directory could be too small to fit all EA's. Usually, if EA's are small enough, file systems like HPFS or JFS can store them in inode/fnode inline. But if they are not fit into inode, they are put into separate streams of sectors. Directory usually only contains a reference to EA's (a first sector/block/cluster number of EA stream, or a handle to a particular EA, or just some bits which denote that EA is present.). Not sure about RISCOS, but AFAIK, MacOS uses a separate stream of data to store its resource forks. Directories ideally should only contain a mapping between inode number and a file name. All standard metadata goes into an inode. But if it's not fit into inode, it goes into separate stream. Inode size is usually limited.
Hence, there is currently no plan to extend Genode's file-system session with extended attributes.
Yes, I of course, understand the wish to keep the interfaces and architecture simpler.
Valery's remark about an emulation scheme looks good to bridge the gap. This way, such a feature can be implemented within the libc where the libc would rely on some conventions. E.g., when asked for the extended attributes for a certain 'file', the libc may - under the hood - read the meta data from a file called '.xa/file'. Yes, this would spill the '.xa' directories here and there but it leaves the complexity out of base mechanisms like the VFS or the file-system session.
Yes, the EA/xattr store can be moved to a separate subdirectories of each current subdirectory (like in svn working copies.). This seems to be a better idea than to have EA's laying around along with their files. So, they will not pollute the usual disk contents. Yes, it looks be possible to have EA support on a libc level. Usually, EA's are returned along with standard file metainfo when setting or retrieving FileInfo/PathInfo or searching files. So, e.g., a file search retrieves a set of records, per each file. These records may contain a standard metainfo (inode contents), as well as EA contents. Also, when opening files, "open" syscall could create EA's for that file. Pointer to EA data is specified as a separate syscall parameter. Though, the open/fopen calls don't contain such parameters, but a native syscall does. Creating a FS-independent wrapper on libc level looks possible, though I fear that it could be inefficient and slow. In a file search example, if a FS returned a set of records containing file metadata, then libc wrapper should process this set of records, and add EA data to each record. This looks to be working, but it could be slow. Usually, a FS does the same with a single step: usually both, metadata contained in inode, and EA's, a returned at once. For speeding things up, the FS doesn't check each file for EA presence, instead, a flag is used, which specifies, whether file has EA's. If not, the EA lookup is just omitted.
WBR, valery
Cheers Norman
Genode users mailing list users@lists.genode.org https://lists.genode.org/listinfo/users