native gcc and external image mount example

[Alexander Tormasov]

	Hi Norman,
thank for comments.
Some questions below

> Once integrating the tool chain in an interactive workflow on top of
> Genode (e.g., as Sculpt subsystem), we'll have to take a closer look of
> course.

ok, good - I just want to be sure that this is not my mistake and misconfiguration


> 
> BTW, I sense from your other posting about the container topic
> (inquiring about chroot, page sharing, notion of the VFS, sharing of
> file descriptors) that you presume Genode to be closer to Unix-like
> systems than it really is. To get a tangible feeling of how Unix
> concepts are realized on Genode, I warmly recommend to grab a tee or
> coffee and enjoy going though the article series referenced at the end
> of the Goa README [1] - in particular the "Sticking together a little
> Unix" episodes should be eye opening.
> 
> [1] https://github.com/nfeske/goa

This is exactly what I does yesterday, when I unable to integratie tcp_terminal into bash scenario.
I want to use is in headless mode, because my build computer is physically far from my MacBook and obviously SDL lib does not works here.
Palliative decision was to always copy/start created iso image to macbook and run qemu locally (where SDL somehow work, as I remember from the past), but more convenient will be plain "remote login» into genode/bash instance with all tools inside.

I think that I can adopt scenario of GUI based terminal implementation (taken from bash.run) by replacement of terminal -> tcp_terminal, and then it does not work.
main problem is here
[init -> tcp_terminal] lwIP Nic interface down
[init -> tcp_terminal] --- TCP terminal started ---
[init] child "tcp_terminal" announces service "Terminal"
…
[init -> nic_drv] MAC address 52:54:00:12:34:56
[init -> tcp_terminal] Warning: no policy defined for label 'vfs -> '
[init -> vfs] Error: Terminal-session creation failed (ram_quota=10240, cap_quota=3, label="")
[init -> vfs] Error: failed to create <terminal> VFS node
[init -> tcp_terminal] lwIP Nic interface up address=10.0.3.2 netmask=255.255.255.0 gateway=10.0.3.1

cryptic message about empty label and a reason for <terminal> failure  is not clear for me, I think that I do provide inside vfs policy for the terminal like 

    <start name="vfs" caps="200">
        <resource name="RAM" quantum="124M"/>
        <provides> <service name="File_system"/> </provides>
        <config>
            <vfs>
                } [mounted_tar_archives] {
                <dir name="dev">
                    <zero/>
                    <null/>
                    <terminal/>
                    <log/>
                    <inline name="rtc">2022-01-01 00:01</inline>
                </dir>
                <dir name="pipe"> <pipe/> </dir>
                <dir name="home">
                    <ram/>
                    <rom name=".bash_profile" label="bash_profile" binary="no"/>
                </dir>
                <dir name="share"> <dir name="vim"> <rom name="vimrc"/> </dir> </dir>
                <dir name="genode"> <tar name="genode.tar"/> </dir>
                <dir name="tmp"> <ram/> </dir>
                <dir name="usr">
                    <dir name="bin">
                        <symlink name="make"  target="/bin/make" />
                        <symlink name="mkdir" target="/bin/mkdir"/>
                        <symlink name="echo"  target="/bin/echo" />
                        <symlink name="tclsh" target="/bin/tclsh"/>
                    </dir>
                </dir>

                <dir name="bin">
                    <symlink name="nm" target="} [prefixed_nm] {"/>
                    <symlink name="sh" target="bash"/>
                </dir>
            </vfs>
            <default-policy root="/" writeable="yes"/>
            <policy label_prefix="vfs_rom" root="/"/>
            <policy label_prefix="/bin/bash" root="/" writeable="yes" />
        </config>
        <route>
            <service name="Terminal"> <child name="tcp_terminal"/> </service>
            <service name="Timer"> <child name="timer"/> </service>
            <any-service> <parent/> </any-service>
        </route>


So, I start reading these 3 articles, and try to re-read foundation book.
Now I understand better while still not able to solve a problem (and I have some comments about understandability of some examples and presented concepts, not all my questions was answered after reading).

May be I do not understand a proper way to integrate terminal service (provided as «Terminal»  by tcp_terminal ), bash (which have libc section in config, related to /dev/terminal), vfs (which label it should announce? and why it will be associated with /dev/terminal in another process…)

I assume that bash process should be connected via /dev/terminal to tcp_terminal which obtain from network configured port a TCP connection. Then I can use telnet localhost 5555 - and this will give me textual console where I can send commands to bash and see it’s output.

may be I can use «terminal» (not tcp_terminal)  without graphics (I see it requires Gui service provided by nitpicker, how it can be replaced?)

Or I mess something and I need to setup ssh_server inside genode?

> 
> You will see that many things work quite differently. Questions like
> memory sharing of read-only binary segments are actually solved problems
> (using cached_fs_rom as a back end for execve).

Seems that I do formulate my questions in a bit wrong way.
For me it is quite obvious that sharing problems should be solvable onto of most of the modern kernels, including the Genode framework with lower level supported kernels.
Main question is a way how I can organise the conditions where particular feature (in this case files sharing in memory for files used from the same instance of file system)  will work, what kind of interfaces I should use.

From the point of view of my nearest goal - ability to run docker container inside genode - I have 2 options.
1. implement runc (low level docker manipulation [1,2]) using plain native Genode interface - to create container and to allow different docker features ontop. This is a hard way, to write things from the scratch.
2. take existing implementations of runc which use native Unix/Windows/Linux/xxxBSD/Hurd/etc OS API and interfaces like /proc to create and manage containers. For me this is a simpler way, I can use current port of Genode libc (because it based on FreeBSD libc sources then I can also start from FreeBSD and, as similar and simpler version NetBSD as a initial point).

Then it is easy to integrate with (written on Golang) docker support code (like containerd + docker cli).
Direct benefit of this approach is that it provide smooth path for integration with most modern docker tools like Kubernetis - they assume docker cli available.

Therefore, when I ask some questions about «processes» (I know that word process is not actively used in genode foundations books, not a problem at all) I mean what kind of interfaces (in case of libc it is from this port, if now - what else I can do) I can choose (from current palettes of options available in the current code base).

example of such question is - whether the single instance of FS could serve different «containers» (assuming that each container is a kind of subsystem , e.g. unix subsystem with forked bunch of processes)? As I understand - yes, it could, and, if different containers open the same file path from the FS server then they will share code pages (at least read-only parts).
For me I see some analogy with having single fs tree serving different containers on normal unix/linux, FS is like a host mount point.
Problem that we want to have different subset of files visible for different instances of containers.
This is related to 2 problems: 
1. own modified files of single containers should be visible only to him, while single FS instance could show them to all clients in theory (CoW private area visibility)
2 every container could start from different set of files (even readonly). Like we can have different linux distribution running and available for different containers, like different versions of glibc or even the whole distribution. To implement this in Virtuozzo and openvz we do implement special templates of file system and applications which could be installed and visible only for particular instances of containers.

both cases above surely assume ability to share files - the problem is organise proper environment (read: sequences of OS api/syscalls/ access to /proc ad so on). Including questions of security, visibility, access rights/etc.


> 
> Given the tool_chain_auto.run scenario with GCC as reference, you may
> consider trying to integrate GCC (and gccgo) into the Unix-like system
> built in the Goa series. This would make a worthwhile exercise.

yes, thanks for suggestion!
while I started from combination of tool_chain_auto.run + tcp_terminal.run inside qemu/nova x86_64, and fail a bit, see above.
probably I need to update it to goa (as I suppose it mostly tailored for linux? not all examples from genode do support linux configuration in networking…)
I definitely don’t want to involve graphics in this stage.

PS the reason why I want to have a native development is quite simple - «go build» do not works in cross mode, typically it immediately build and run something locally. Golang do create their own version of «dll hell» using vendors version of packages, and «go build» is the only reasonable way to survive...

[1] https://www.docker.com/blog/runc/
[2] https://github.com/opencontainers/runc