Hello
I created a vfs package archive containing vfs and vfs_cow and start it within the sculpt runtime providing the following configuration:
<config> <vfs> <dir name="immutable"> <fs label="template_data"/> </dir> <dir name="mutable"> <fs label="user_data"/> </dir> <dir name="cow"> <cow ro="/immutable" rw="/mutable"/> </dir> </vfs> <default-policy root="/cow" writeable="yes"/> </config>
Then, I changed the "target" file system of the noux package to use the above copy-on-write file system to play around with it...
First thing I noticed is that the directories and files of "template_data" are not populated. I guess that's a feature that is not yet or will not be implemented and for my use case it doesn't bother me right now.
But when I open/write a file (that exists in template_data), cow_vfs writes the following output:
[runtime -> cow_fs] Warning: COW: block for replication... [runtime -> cow_fs] Warning: COW: block for replication...
...and never unblocks afterwards. If immutable is something like </ram> or </inline ...>, it works as I would expect it. Is this a known limitation? Or can someone point me to the right direction to solve it?
I love the idea of the vfs_cow component and I guess it may cover many other use cases!
Cheers, Roman
Hello Roman,
Yes, the COW plugin has some known and expected limitations, and thats my fault. Replicating across immediate file-systems like ram, tar, or inline should work fine, but replication across the File_system session is problematic. The issue here is that chaining internal asynchronous reads and writes is poorly handled when the plugin is used with the VFS server or internally in Noux.
At the moment the COW plugin is really a simple 'lazy replication' plugin. True copy-on-write behavior seems to to requires some caching and state tracking, which was something that was avoided in this early version. I hope in the next few weeks to move replication to a background task to improve performance, and that will eliminate the 'block for replication' condition.
Cheers, Emery
On Fri, 15 Jun 2018 16:53:15 +0200 Roman Iten roman.iten@gapfruit.com wrote:
Hello
I created a vfs package archive containing vfs and vfs_cow and start it within the sculpt runtime providing the following configuration:
<config> <vfs> <dir name="immutable"> <fs label="template_data"/> </dir> <dir name="mutable"> <fs label="user_data"/> </dir> <dir name="cow"> <cow ro="/immutable" rw="/mutable"/> </dir> </vfs> <default-policy root="/cow" writeable="yes"/> </config>
Then, I changed the "target" file system of the noux package to use the above copy-on-write file system to play around with it...
First thing I noticed is that the directories and files of "template_data" are not populated. I guess that's a feature that is not yet or will not be implemented and for my use case it doesn't bother me right now.
But when I open/write a file (that exists in template_data), cow_vfs writes the following output:
[runtime -> cow_fs] Warning: COW: block for replication... [runtime -> cow_fs] Warning: COW: block for replication...
...and never unblocks afterwards. If immutable is something like </ram> or </inline ...>, it works as I would expect it. Is this a known limitation? Or can someone point me to the right direction to solve it?
I love the idea of the vfs_cow component and I guess it may cover many other use cases!
Cheers, Roman
Hello Roman,
There is now an "import" VFS plugin that similiar to the COW plugin, but with a clearer concept and implmentation. The import plugin will simply recursively copy its contents to the root directory of the VFS that is is configured in, before the "application" becomes active. The plugin is available on the genodelabs/staging branch and we be used in the next Sculpt release.
You may refer to "sculpt.run" on staging as a configuration reference: https://github.com/genodelabs/genode/blob/1b4ccc854f0b6226273ed7b881b8ac30e2...
Hope thats helpful, Emery
On Saturday, June 16, 2018 11:14:50 AM CEST, Emery Hemingway wrote:
Hello Roman,
Yes, the COW plugin has some known and expected limitations, and thats my fault. Replicating across immediate file-systems like ram, tar, or inline should work fine, but replication across the File_system session is problematic. The issue here is that chaining internal asynchronous reads and writes is poorly handled when the plugin is used with the VFS server or internally in Noux.
At the moment the COW plugin is really a simple 'lazy replication' plugin. True copy-on-write behavior seems to to requires some caching and state tracking, which was something that was avoided in this early version. I hope in the next few weeks to move replication to a background task to improve performance, and that will eliminate the 'block for replication' condition.
Cheers, Emery
On Fri, 15 Jun 2018 16:53:15 +0200 Roman Iten roman.iten@gapfruit.com wrote:
Hello
I created a vfs package archive containing vfs and vfs_cow and start it within the sculpt runtime providing the following configuration:
<config> <vfs> <dir name="immutable"> <fs label="template_data"/> </dir> <dir name="mutable"> <fs label="user_data"/> </dir> <dir name="cow"> <cow ro="/immutable" rw="/mutable"/> </dir> </vfs> <default-policy root="/cow" writeable="yes"/> </config>
Then, I changed the "target" file system of the noux package to use the above copy-on-write file system to play around with it...
First thing I noticed is that the directories and files of "template_data" are not populated. I guess that's a feature that is not yet or will not be implemented and for my use case it doesn't bother me right now.
But when I open/write a file (that exists in template_data), cow_vfs writes the following output:
[runtime -> cow_fs] Warning: COW: block for replication... [runtime -> cow_fs] Warning: COW: block for replication...
...and never unblocks afterwards. If immutable is something like </ram> or </inline ...>, it works as I would expect it. Is this a known limitation? Or can someone point me to the right direction to solve it?
I love the idea of the vfs_cow component and I guess it may cover many other use cases!
Cheers, Roman
Hi Emery
On 07/26/2018 02:16 PM, Emery Hemingway wrote:
There is now an "import" VFS plugin that similiar to the COW plugin, but with a clearer concept and implmentation. The import plugin will simply recursively copy its contents to the root directory of the VFS that is is configured in, before the "application" becomes active. The plugin is available on the genodelabs/staging branch and we be used in the next Sculpt release.
Thanks, I didn't actually try it yet but it looks like it will indeed cover my use case perfectly! Just one comprehension question regarding the following configuration:
! <config> ! <vfs> ! <fs label="user_data"/> ! <import> ! <fs label="template_data"> ! </import> ! </vfs> ! <default-policy root="/" writeable="yes"/> ! </config>
In case "user_data" is a persistent file system, the files and/or directories that are going to be imported from "template_data" may already exist in "user_data". Will they remain untouched or are they overwritten? Or is there an attribute (like <import overwrite|force=...) to control the behavior?
Thanks, Roman
On Saturday, June 16, 2018 11:14:50 AM CEST, Emery Hemingway wrote:
Hello Roman,
Yes, the COW plugin has some known and expected limitations, and thats my fault. Replicating across immediate file-systems like ram, tar, or inline should work fine, but replication across the File_system session is problematic. The issue here is that chaining internal asynchronous reads and writes is poorly handled when the plugin is used with the VFS server or internally in Noux.
At the moment the COW plugin is really a simple 'lazy replication' plugin. True copy-on-write behavior seems to to requires some caching and state tracking, which was something that was avoided in this early version. I hope in the next few weeks to move replication to a background task to improve performance, and that will eliminate the 'block for replication' condition.
Cheers, Emery
On Fri, 15 Jun 2018 16:53:15 +0200 Roman Iten roman.iten@gapfruit.com wrote:
Hello
I created a vfs package archive containing vfs and vfs_cow and start it within the sculpt runtime providing the following configuration:
<config> <vfs> <dir name="immutable"> <fs label="template_data"/> </dir> <dir name="mutable"> <fs label="user_data"/> </dir> <dir name="cow"> <cow ro="/immutable" rw="/mutable"/> </dir> </vfs> <default-policy root="/cow" writeable="yes"/> </config>
Then, I changed the "target" file system of the noux package to use the above copy-on-write file system to play around with it...
First thing I noticed is that the directories and files of "template_data" are not populated. I guess that's a feature that is not yet or will not be implemented and for my use case it doesn't bother me right now.
But when I open/write a file (that exists in template_data), cow_vfs writes the following output:
[runtime -> cow_fs] Warning: COW: block for replication... [runtime -> cow_fs] Warning: COW: block for replication...
...and never unblocks afterwards. If immutable is something like </ram> or </inline ...>, it works as I would expect it. Is this a known limitation? Or can someone point me to the right direction to solve it?
I love the idea of the vfs_cow component and I guess it may cover many other use cases!
Cheers, Roman
Hi Roman,
Good to hear that this will work for you, the "import" use case seems much more common than a full "copy-on-write".
On Monday, July 30, 2018 3:02:09 PM CEST, Roman Iten wrote:
! <config> ! <vfs> ! <fs label="user_data"/> ! <import> ! <fs label="template_data"> ! </import> ! </vfs> ! <default-policy root="/" writeable="yes"/> ! </config>
In case "user_data" is a persistent file system, the files and/or directories that are going to be imported from "template_data" may already exist in "user_data". Will they remain untouched or are they overwritten? Or is there an attribute (like <import overwrite|force=...) to control the behavior?
On the "genodelabs/staging" branch the plugin will never overwrite existing files, but on the "cnuke/staging" branch there is a commit that adds an `<import overwrite="1">` option.
https://github.com/cnuke/genode/commit/332a9d9382e776fa5385331fef1750abcc0a6...
Cheers, Emery