Hi,
I have been looking at several drivers in Genode, and they all seem to have static data - specifically a Heap object and Root object. I am wondering with this kind of setup, is it possible to manage more than one device with a single driver?
Thanks, Joel
Hello,
On 23.10.18 14:50, Joel Nider wrote:
I have been looking at several drivers in Genode, and they all seem to have static data - specifically a Heap object and Root object. I am wondering with this kind of setup, is it possible to manage more than one device with a single driver?
of course. As described in the Genode book [0] in chapter 3.2.3, the root object provides the implementation just of the interface. Whenever a client connects to a service (like a driver for nic, ahci, nvme, usb etc.) a session is established via this interface. The driver may associate (typically does so depending on your specified policy) one specific device it drives to this session - as done e.g. for drivers handling multiple devices, e.g. ahci, nvme, usb.
A driver may provide also different services, e.g. our old usb driver provides Nic, Block, Input (so several root interfaces).
Cheers,
Alex.
[0] https://www.genode.org/documentation/genode-foundations-18-05.pdf
Hi Alex,
On 23.10.18 14:50, Joel Nider wrote:
I have been looking at several drivers in Genode, and they all seem to
have static data - specifically a Heap object and Root object. I am wondering with this kind of setup, is it possible to manage more than
one
device with a single driver?
of course. As described in the Genode book [0] in chapter 3.2.3, the root object provides the implementation just of the interface. Whenever a client connects to a service (like a driver for nic, ahci, nvme, usb etc.) a session is established via this interface. The driver may associate (typically does so depending on your specified policy) one specific device it drives to this session - as done e.g. for drivers handling multiple devices, e.g. ahci, nvme, usb.
I will give you my situation as a specific example. I am writing a driver for a PCIe attached NIC (x86 platform). Let us say that I have 2 physical NICs in my machine with the same device ID and vendor ID. I have specified in my config file that I have a driver called "nic_drv", without specifying the PCIe address (copied from the example in repos/os/run/ping.run).
When the driver starts, I can enumerate the PCIe devices through the Platform::Connection object that is instantiated from the 'env' object. I see all PCIe devices of class 'NET' (even other NICs that I don't want my driver to handle). So far so good. I can filter through the devices by requesting the capability (with Platform::Device_client) and I can read the device ID and vendor ID and pick the devices I want to handle.
Now I have a list of several devices that I want my driver handle. If I understand you correctly, I should have only one Root object (and one Heap object). That implies I should also only call parent().announce() once. But then how do I instantiate my Nic::Session_component once per device? What is happening so far, is that some time after I call parent().announce(), my Nic::Session_component derived class gets instantiated, and the constructor is called. But I have no way of knowing which device it is meant to handle, since there is no mechanism for passing the device capability to Nic::Session_component. In addition, I only see this constructor being called once. I expected to see one constructor per PCIe device. This causes further problems such as not being able to map the mmio region since the size and base address are not known until I query the PCIe BAR, and there is no 'new' operator available, which means to me I'm doing something fundamentally wrong.
So what is the correct way to build the driver such that I get an object per device (and the object knows the device)?
Thanks, Joel
On 25.10.18 09:38, Joel Nider wrote:
On 23.10.18 14:50, Joel Nider wrote:
I have been looking at several drivers in Genode, and they all seem to
have static data - specifically a Heap object and Root object. I am wondering with this kind of setup, is it possible to manage more than
one
device with a single driver?
of course. As described in the Genode book [0] in chapter 3.2.3, the root object provides the implementation just of the interface. Whenever a client connects to a service (like a driver for nic, ahci, nvme, usb etc.) a session is established via this interface. The driver may associate (typically does so depending on your specified policy) one specific device it drives to this session - as done e.g. for drivers handling multiple devices, e.g. ahci, nvme, usb.
I will give you my situation as a specific example. I am writing a driver for a PCIe attached NIC (x86 platform). Let us say that I have 2 physical NICs in my machine with the same device ID and vendor ID. I have specified in my config file that I have a driver called "nic_drv", without specifying the PCIe address (copied from the example in repos/os/run/ping.run).
When the driver starts, I can enumerate the PCIe devices through the Platform::Connection object that is instantiated from the 'env' object. I see all PCIe devices of class 'NET' (even other NICs that I don't want my driver to handle). So far so good. I can filter through the devices by requesting the capability (with Platform::Device_client) and I can read the device ID and vendor ID and pick the devices I want to handle.
Actually, in this case, I would not encourage to implement it in one driver. The network cards are independent (as opposed for devices on a ahci controller) from each other, so just start/instantiate the driver two times. With that you cleanly isolate the network data.
For that to work, you have to configure the platform driver (the guard to the PCIe bus on Genode) [0] to hand out the first driver one network card (described as PCI bus device function) and for the second driver the other network card (described by another PCI bus device function).
[0] https://github.com/genodelabs/genode/blob/master/repos/os/src/drivers/platfo...
Now I have a list of several devices that I want my driver handle. If I understand you correctly, I should have only one Root object (and one Heap object). That implies I should also only call parent().announce() once. But then how do I instantiate my Nic::Session_component once per device? What is happening so far, is that some time after I call parent().announce(), my Nic::Session_component derived class gets instantiated, and the constructor is called. But I have no way of knowing which device it is meant to handle, since there is no mechanism for passing the device capability to Nic::Session_component. In addition, I only see this constructor being called once. I expected to see one constructor per PCIe device. This causes further problems such as not being able to map the mmio region since the size and base address are not known until I query the PCIe BAR, and there is no 'new' operator available, which means to me I'm doing something fundamentally wrong.
So what is the correct way to build the driver such that I get an object per device (and the object knows the device)?
As said, I would not go further this route in your case, since what you are asking for does not match your use-case.
Nevertheless, it is possible, as example look into the ahci driver [1]. In short, when a client calls you, the _create_session method is called, there you need some differentiator to decide, which session gets which device.
[1] repos/os/src/drivers/ahci/main.cc
In the ahci driver case one has to configure it by a label and a device argument in the policy. Analogously, for the network driver it could hypothetical look like this :
<start name="my_nic_drv"> ... <provides><service name="Nic"/></provides> <config> <policy label="network-black" device="1" /> <policy label="network-red" device="2" /> </config> </start>
<start name="client-a"> ... <route> ... <service name="Nic"> <child name="my_nic_drv" label="network-black"/> </service </route> </start>
<start name="client-b"> ... <route> ... <service name="Nic"> <child name="my_nic_drv" label="network-red"/> </service </route> </start>
In your _create_session method implementation you would evaluate for the device arguments and knows which device to which client belongs.
Further, as you can see in the ahci driver, you can of course use 'new' if you specify a allocator.
Hope it helps,
Alex.
Hi,
On Thu, Oct 25, 2018 at 02:16:35PM +0200, Alexander Boettcher wrote:
Actually, in this case, I would not encourage to implement it in one driver. The network cards are independent (as opposed for devices on a ahci controller) from each other, so just start/instantiate the driver two times. With that you cleanly isolate the network data.
For that to work, you have to configure the platform driver (the guard to the PCIe bus on Genode) [0] to hand out the first driver one network card (described as PCI bus device function) and for the second driver the other network card (described by another PCI bus device function).
I actually did such a two-NICs scenario a while ago.
https://github.com/chelmuth/genode/blob/4cb534662b2c52c3de726eedc44664916635...
Regards
Hi Christian,
Christian Helmuth christian.helmuth@genode-labs.com wrote on 25/10/2018 15:41:14:
From: Christian Helmuth christian.helmuth@genode-labs.com To: users@lists.genode.org Date: 25/10/2018 15:44 Subject: Re: multiple devices, single driver
Hi,
On Thu, Oct 25, 2018 at 02:16:35PM +0200, Alexander Boettcher wrote:
Actually, in this case, I would not encourage to implement it in one driver. The network cards are independent (as opposed for devices on a ahci controller) from each other, so just start/instantiate the driver two times. With that you cleanly isolate the network data.
For that to work, you have to configure the platform driver (the guard to the PCIe bus on Genode) [0] to hand out the first driver one
network
card (described as PCI bus device function) and for the second driver the other network card (described by another PCI bus device function).
I actually did such a two-NICs scenario a while ago.
Thanks for the example - this is very clear. However, its also very hard-coded. Is there a way to support the two-NIC scenario without specifying the PCIe address? For example, if I was using PCIe hot plug, or was just too lazy to reboot several times looking for the bus ID and re-programming the config file?
INVALID URI REMOVED
u=https-3A__github.com_chelmuth_genode_blob_4cb534662b2c52c3de726eedc4466491663523d7_repos_libports_run_twonics.run-23L40-2DL44&d=DwIGaQ&c=jf_iaSHvJObTbx-
siA1ZOg&r=odoeHilLk1wCp9UNyGcJLQ&m=14Y8IwKJpMUWTgOzRcJ8yiuUqw32UVu1edCxGCjA1fc&s=sr4VhiOSFg_l72I_3YICDU6hh4bP0qVmyBNhj6tQTlw&e=
Regards
Christian Helmuth Genode Labs
INVALID URI REMOVED u=https-3A__www.genode-2Dlabs.com_&d=DwIGaQ&c=jf_iaSHvJObTbx-
siA1ZOg&r=odoeHilLk1wCp9UNyGcJLQ&m=14Y8IwKJpMUWTgOzRcJ8yiuUqw32UVu1edCxGCjA1fc&s=nPqO5RSRIMEoI-
DXLw8qRzqgdj-ONFlSQxIqCKVoGno&e= ·
INVALID URI REMOVED
u=https-3A__genode.org_&d=DwIGaQ&c=jf_iaSHvJObTbx-
siA1ZOg&r=odoeHilLk1wCp9UNyGcJLQ&m=14Y8IwKJpMUWTgOzRcJ8yiuUqw32UVu1edCxGCjA1fc&s=DyMUWEOGn1D-
WmBsv247oQAQ-g1H2ZIE6Yeiw3qrJew&e= INVALID URI REMOVED u=https-3A__twitter.com_GenodeLabs&d=DwIGaQ&c=jf_iaSHvJObTbx-
siA1ZOg&r=odoeHilLk1wCp9UNyGcJLQ&m=14Y8IwKJpMUWTgOzRcJ8yiuUqw32UVu1edCxGCjA1fc&s=BJhQxlY6P8rwH54ow5mKoD5_4tpzX5iXU8pBMbAqJbw&e=
· /?d?i?.n??d/
Genode Labs GmbH · Amtsgericht Dresden · HRB 28424 · Sitz Dresden Geschäftsführer: Dr.-Ing. Norman Feske, Christian Helmuth
Genode users mailing list users@lists.genode.org INVALID URI REMOVED u=https-3A__lists.genode.org_listinfo_users&d=DwIGaQ&c=jf_iaSHvJObTbx-
siA1ZOg&r=odoeHilLk1wCp9UNyGcJLQ&m=14Y8IwKJpMUWTgOzRcJ8yiuUqw32UVu1edCxGCjA1fc&s=hmPN_vKfLWhDpG1ImnNNIeqH3gtwxVJhcCRwFRS_mE0&e=
Hi Alex,
Alexander Boettcher alexander.boettcher@genode-labs.com wrote on 25/10/2018 15:16:35:
From: Alexander Boettcher alexander.boettcher@genode-labs.com To: Joel Nider JOELN@il.ibm.com Cc: Genode users mailing list users@lists.genode.org Date: 25/10/2018 15:16 Subject: Re: multiple devices, single driver
On 25.10.18 09:38, Joel Nider wrote:
On 23.10.18 14:50, Joel Nider wrote:
I have been looking at several drivers in Genode, and they all seem
to
have static data - specifically a Heap object and Root object. I am
wondering with this kind of setup, is it possible to manage more
than
one
device with a single driver?
of course. As described in the Genode book [0] in chapter 3.2.3, the root object provides the implementation just of the interface.
Whenever
a client connects to a service (like a driver for nic, ahci, nvme,
usb
etc.) a session is established via this interface. The driver may associate (typically does so depending on your specified policy) one specific device it drives to this session - as done e.g. for drivers handling multiple devices, e.g. ahci, nvme, usb.
I will give you my situation as a specific example. I am writing a
driver
for a PCIe attached NIC (x86 platform). Let us say that I have 2
physical
NICs in my machine with the same device ID and vendor ID. I have
specified
in my config file that I have a driver called "nic_drv", without specifying the PCIe address (copied from the example in repos/os/run/ping.run).
When the driver starts, I can enumerate the PCIe devices through the Platform::Connection object that is instantiated from the 'env'
object. I
see all PCIe devices of class 'NET' (even other NICs that I don't want
my
driver to handle). So far so good. I can filter through the devices by
requesting the capability (with Platform::Device_client) and I can
read
the device ID and vendor ID and pick the devices I want to handle.
Actually, in this case, I would not encourage to implement it in one driver. The network cards are independent (as opposed for devices on a ahci controller) from each other, so just start/instantiate the driver two times. With that you cleanly isolate the network data.
Does that mean the memory is duplicated? Two code segments, two execution threads, two address spaces?
For that to work, you have to configure the platform driver (the guard to the PCIe bus on Genode) [0] to hand out the first driver one network card (described as PCI bus device function) and for the second driver the other network card (described by another PCI bus device function).
[0] INVALID URI REMOVED
u=https-3A__github.com_genodelabs_genode_blob_master_repos_os_src_drivers_platform_spec_x86_README&d=DwIDaQ&c=jf_iaSHvJObTbx-
siA1ZOg&r=odoeHilLk1wCp9UNyGcJLQ&m=nPjNOfZv-
iOtQesvBgSv3JPpyFF9cGnynFRPbt7USbI&s=vP_rZKipGOq1_HBljQxfPIqBxQ_enfDC7Qv2zZKpbaI&e=
Now I have a list of several devices that I want my driver handle. If
I
understand you correctly, I should have only one Root object (and one
Heap
object). That implies I should also only call parent().announce()
once.
But then how do I instantiate my Nic::Session_component once per
device?
What is happening so far, is that some time after I call parent().announce(), my Nic::Session_component derived class gets instantiated, and the constructor is called. But I have no way of
knowing
which device it is meant to handle, since there is no mechanism for passing the device capability to Nic::Session_component. In addition,
I
only see this constructor being called once. I expected to see one constructor per PCIe device. This causes further problems such as not being able to map the mmio region since the size and base address are
not
known until I query the PCIe BAR, and there is no 'new' operator available, which means to me I'm doing something fundamentally wrong.
So what is the correct way to build the driver such that I get an
object
per device (and the object knows the device)?
As said, I would not go further this route in your case, since what you are asking for does not match your use-case.
Nevertheless, it is possible, as example look into the ahci driver [1]. In short, when a client calls you, the _create_session method is called, there you need some differentiator to decide, which session gets which device.
[1] repos/os/src/drivers/ahci/main.cc
In the ahci driver case one has to configure it by a label and a device argument in the policy. Analogously, for the network driver it could hypothetical look like this :
<start name="my_nic_drv"> ... <provides><service name="Nic"/></provides> <config> <policy label="network-black" device="1" /> <policy label="network-red" device="2" /> </config> </start>
<start name="client-a"> ... <route> ... <service name="Nic"> <child name="my_nic_drv" label="network-black"/> </service </route> </start>
<start name="client-b"> ... <route> ... <service name="Nic"> <child name="my_nic_drv" label="network-red"/> </service </route> </start>
In your _create_session method implementation you would evaluate for the device arguments and knows which device to which client belongs.
Further, as you can see in the ahci driver, you can of course use 'new' if you specify a allocator.
Hope it helps,
Alex.
-- Alexander Boettcher Genode Labs
INVALID URI REMOVED u=https-3A__www.genode-2Dlabs.com&d=DwIDaQ&c=jf_iaSHvJObTbx- siA1ZOg&r=odoeHilLk1wCp9UNyGcJLQ&m=nPjNOfZv-
iOtQesvBgSv3JPpyFF9cGnynFRPbt7USbI&s=YVsmmPBESSsyKqhHPmeYxHPi7K--MY7ErmoMyLCkqiI&e= -
INVALID URI REMOVED u=https-3A__www.genode.org&d=DwIDaQ&c=jf_iaSHvJObTbx- siA1ZOg&r=odoeHilLk1wCp9UNyGcJLQ&m=nPjNOfZv-
iOtQesvBgSv3JPpyFF9cGnynFRPbt7USbI&s=_LZwjrY1hEP7AE46D9sU7jISm0EohYKXZJUVfskmQPQ&e=
Genode Labs GmbH - Amtsgericht Dresden - HRB 28424 - Sitz Dresden Geschäftsführer: Dr.-Ing. Norman Feske, Christian Helmuth
Hi Joel
On 25.10.18 17:14, Joel Nider wrote:
I will give you my situation as a specific example. I am writing a
driver
for a PCIe attached NIC (x86 platform). Let us say that I have 2
physical
NICs in my machine with the same device ID and vendor ID. I have
specified
in my config file that I have a driver called "nic_drv", without specifying the PCIe address (copied from the example in repos/os/run/ping.run).
When the driver starts, I can enumerate the PCIe devices through the Platform::Connection object that is instantiated from the 'env'
object. I
see all PCIe devices of class 'NET' (even other NICs that I don't want
my
driver to handle). So far so good. I can filter through the devices by
requesting the capability (with Platform::Device_client) and I can
read
the device ID and vendor ID and pick the devices I want to handle.
Actually, in this case, I would not encourage to implement it in one driver. The network cards are independent (as opposed for devices on a ahci controller) from each other, so just start/instantiate the driver two times. With that you cleanly isolate the network data.
Does that mean the memory is duplicated? Two code segments, two execution threads, two address spaces?
Yes. Effectively you can trade memory(+bit of CPU time) with isolation and robustness.
Imagine your driver is handling 8 (or more) identical network cards, and the driver fails for some reason. Your 8 network links are down until you restart the driver on the fly.
In contrast, if you have 8 network driver instances, one failure of one driver has no effect to the other 7 network links. The downtime of the other links are -zero- and you have just to bring up one network card driver again. Additionally, no network traffic can "accidentally" leak between the network links. Furthermore, if required, the placement of the multiple driver instances on specific cores (e.g. pinning) becomes obvious - and easy.
At the end, I think that the driver code will become much easier to read and to grasp and this will pay off later during maintenance time.
Of course, if you are running in a embedded setup and can't effort, say less than 1M per driver instance, maybe the decision is different.
Cheers,
Alex.
-
Alexander Boettcher -
Christian Helmuth -
Joel Nider