I suppose we could use this mechanism, which we already use for ioctl, for several other file attributes as well, e.g. timestamps. We could put it all under a directory named .attrs-(filename).

As for performance, the filesystem drivers could implement attribute access as separate files while still using attrs/xattrs as the underlying storage mechanism. The main issue with this is when attributes are copied before the files. However, if we make the attributes transparent in libc, this shouldn't be much of an issue, especially if we guarantee that the attributes come after the files in the dirent listing. Overall, I think this could be a relatively clean way to implement both regular and extended attributes. Any other thoughts on this?

On Tue, Apr 2, 2019, 7:54 AM Valery V. Sedletski via users <users@lists.genode.org> wrote:
Norman Feske wrote:
> Hello Valery,
>
> thanks for the background info about extended attributes.
>
>> mandatory to be existed when opening the file). So, EA's on FAT32 are
>> laying around near the file itself, in the same directory, in a hidden
>> file. The disadvantage is that EA files are laying around everywhere,
>> like a trash. So, if a file system doesn't support EA's, they can be
>> emulated like this.
> I admittedly have not much clue about file systems. To me as a user,
> features like this (this also goes for the strange "resource forks" of
> old Mac OS) look like glorified directories. They seem to be just more
> complicated, less transparent, and cause friction when moving data
> across file systems. Given my poor background, please take the following
> with a grain of salt.
Yes, also WinNT goes even further: it supports files having multiple
streams of data. Resource fork from MacOS or EA can be implemented
as streams too. Only Resource fork stores multiple metainfo in one
stream, and EA's can be implemented both ways: as a single stream,
or multiple streams.
> I wonder, in situations where meta data has to contain a lot of domain
> knowledge, wouldn't sqlite be a good way to organize it?
Probably yes, though, sqlite files are just tables with fixed number
of fields. Storing EA in SQLite or any other relational database should
require a data schema with multiple tables, which may be too complex,
because a file can have any number of EA's, whereas a table contains
a fixed number of fields. So, it may require a complicated data schema.
> In situations where meta data are simple annotations to files (like
> storing the icon for a file), why can't both the file and the meta data
> go to into a directory? This is the way Risc OS (and also today's Mac
> OS, AFAIK) deal with the situation.
They could be not simple annotations/text data, but they also could
be any small binary data attached to a file (icons being a good example).

Directory could be too small to fit all EA's. Usually, if EA's are small
enough, file systems like HPFS or JFS can store them in inode/fnode
inline. But if they are not fit into inode, they are put into separate
streams of sectors. Directory usually only contains a reference to EA's
(a first sector/block/cluster number of EA stream, or a handle to a
particular EA, or just some bits which denote that EA is present.).
Not sure about RISCOS, but AFAIK, MacOS uses a separate stream
of data to store its resource forks. Directories ideally should only
contain a mapping between inode number and a file name. All standard
metadata goes into an inode. But if it's not fit into inode, it goes into
separate stream. Inode size is usually limited.
> Hence, there is currently no plan to extend Genode's file-system session
> with extended attributes.
Yes, I of course, understand the wish to keep the interfaces and
architecture simpler.
> Valery's remark about an emulation scheme looks good to bridge the gap.
> This way, such a feature can be implemented within the libc where the
> libc would rely on some conventions. E.g., when asked for the extended
> attributes for a certain 'file', the libc may - under the hood - read
> the meta data from a file called '.xa/file'. Yes, this would spill the
> '.xa' directories here and there but it leaves the complexity out of
> base mechanisms like the VFS or the file-system session.
Yes, the EA/xattr store can be moved to a separate subdirectories
of each current subdirectory (like in svn working copies.). This
seems to be a better idea than to have EA's laying around along
with their files. So, they will not pollute the usual disk contents.
Yes, it looks be possible to have EA support on a libc level.
Usually, EA's are returned along with standard file metainfo
when setting or retrieving FileInfo/PathInfo or searching files.
So, e.g., a file search retrieves a set of records, per each file. These
records may contain a standard metainfo (inode contents), as well
as EA contents. Also, when opening files, "open" syscall could create
EA's for that file. Pointer to EA data is specified as a separate syscall
parameter. Though, the open/fopen calls don't contain such
parameters, but a native syscall does. Creating a FS-independent
wrapper on libc level looks possible, though I fear that it could
be inefficient and slow. In a file search example, if a FS returned
a set of records containing file metadata, then libc wrapper should
process this set of records, and add EA data to each record. This
looks to be working, but it could be slow. Usually, a FS does the same
with a single step: usually both, metadata contained in inode, and EA's,
a returned at once. For speeding things up, the FS doesn't check
each file for EA presence, instead, a flag is used, which specifies,
whether file has EA's. If not, the EA lookup is just omitted.

WBR,
valery

 > Cheers
 > Norman

_______________________________________________
Genode users mailing list
users@lists.genode.org
https://lists.genode.org/listinfo/users