Hi, Sorry for the dumb question, but how do I use the std c++ new placement operator which take void *, i.e.
void * p = ::malloc(256); T * obj = new (p) T();
When I try to use this it cannot find the operator, only the Genode allocator version. I have 'cxx' in my target.mk
Thanks Daniel
Hello Daniel,
[...] how do I use the std c++ new placement operator which take void *, i.e.
void * p = ::malloc(256); T * obj = new (p) T();
When I try to use this it cannot find the operator, only the Genode allocator version. I have 'cxx' in my target.mk
in the rare cases where the Genode base system needs a placement new operator, we host it in the implementation file. For example, 'base/src/base/allocator/slab.cc' contains one. This way, the operator is defined only if really intended, which avoids using it by accident. It is a single line of code:
inline void *operator new(size_t, void *at) { return at; }
Alternatively, if you are using the standard C++ library anyway, you may use the definition provided by libsupc++. Just specify your LIBS declaration in your 'target.mk' file as follows:
LIBS = libc libm stdcxx
Now, you can just '#include <new>' and use the placement new operator provided there.
Cheers Norman
Ahh, thanks Norman, it was the combination of LIBS += stdcxx and #include <new> that I was looking for.
Daniel
On 03/20/2013 12:46 AM, Norman Feske wrote:
Hello Daniel,
[...] how do I use the std c++ new placement operator which take void *, i.e.
void * p = ::malloc(256); T * obj = new (p) T();
When I try to use this it cannot find the operator, only the Genode allocator version. I have 'cxx' in my target.mk
in the rare cases where the Genode base system needs a placement new operator, we host it in the implementation file. For example, 'base/src/base/allocator/slab.cc' contains one. This way, the operator is defined only if really intended, which avoids using it by accident. It is a single line of code:
inline void *operator new(size_t, void *at) { return at; }
Alternatively, if you are using the standard C++ library anyway, you may use the definition provided by libsupc++. Just specify your LIBS declaration in your 'target.mk' file as follows:
LIBS = libc libm stdcxx
Now, you can just '#include <new>' and use the placement new operator provided there.
Cheers Norman
Dear, Norman Feske!
I am sorry for off-topic. Is it possible to use STL containers using mentioned way? (change target.mk) And one more question: how can one use STL containers with Genode new placement operator?
Thanks in advance for your answers. Kind regards, Sergey.
20.03.2013, 11:47, "Norman Feske" <norman.feske@...1...>:
Hello Daniel,
[...] how do I use the std c++ new placement operator which take void *, i.e.
void * p = ::malloc(256); T * obj = new (p) T();
When I try to use this it cannot find the operator, only the Genode allocator version. I have 'cxx' in my target.mk
in the rare cases where the Genode base system needs a placement new operator, we host it in the implementation file. For example, 'base/src/base/allocator/slab.cc' contains one. This way, the operator is defined only if really intended, which avoids using it by accident. It is a single line of code:
inline void *operator new(size_t, void *at) { return at; }
Alternatively, if you are using the standard C++ library anyway, you may use the definition provided by libsupc++. Just specify your LIBS declaration in your 'target.mk' file as follows:
LIBS = libc libm stdcxx
Now, you can just '#include <new>' and use the placement new operator provided there.
Cheers Norman
-- Dr.-Ing. Norman Feske Genode Labs
http://www.genode-labs.com · http://genode.org
Genode Labs GmbH · Amtsgericht Dresden · HRB 28424 · Sitz Dresden Geschäftsführer: Dr.-Ing. Norman Feske, Christian Helmuth
Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_mar _______________________________________________ Genode-main mailing list Genode-main@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/genode-main
Hello Sergey,
I am sorry for off-topic. Is it possible to use STL containers using mentioned way? (change target.mk)
yes. In order to use the C++ standard library, you will need to prepare it first. To do that, just issue the following command from within the 'libports' repository:
make prepare PKG=stdcxx
This will download and integrate the library with Genode. Also make sure that the 'libports' repository is specified in your build directory's 'REPOSITORIES' declaration (in '<build-dir>/etc/build.conf').
Please note, however, that we at Genode Labs don't use the C++ standard library much. So the stdcxx library does not receive much testing from our side. If you encounter problems with it, please bear with us. Also, some modern bits such as C++11 concurrency features are not supported. The containers should work without problems though.
And one more question: how can one use STL containers with Genode new placement operator?
I cannot help you here because I lack proficiency with STL. Maybe Daniel can help out here?
Best regards Norman
Hi Sergey,
W.R.T. Genode allocators and STL - I would of thought that you would not need to do this directly, but that the Genode stdc++ port backs onto a port of malloc, that in turn backs onto Genode allocator calls.
Is there something specific you are trying to do?
Daniel
On 03/21/2013 12:30 AM, Norman Feske wrote:
Hello Sergey,
I am sorry for off-topic. Is it possible to use STL containers using mentioned way? (change target.mk)
yes. In order to use the C++ standard library, you will need to prepare it first. To do that, just issue the following command from within the 'libports' repository:
make prepare PKG=stdcxx
This will download and integrate the library with Genode. Also make sure that the 'libports' repository is specified in your build directory's 'REPOSITORIES' declaration (in '<build-dir>/etc/build.conf').
Please note, however, that we at Genode Labs don't use the C++ standard library much. So the stdcxx library does not receive much testing from our side. If you encounter problems with it, please bear with us. Also, some modern bits such as C++11 concurrency features are not supported. The containers should work without problems though.
And one more question: how can one use STL containers with Genode new placement operator?
I cannot help you here because I lack proficiency with STL. Maybe Daniel can help out here?
Best regards Norman
Hello, Daniel! Thanks for your answer!
My interest about STL containers has quite simple prerequisite. As far as I know Genode implementations of data structure (particularly, Avl_tree) has some issues and Genodelabs does not recommend it for usage in user applications. The reasons that issues are still not resolved is quite serious: Genode is kept as simple as possible. So, I was thinking about making a kind of porting of STL to Genode. But, it seems that this work is redundant since it is possible to use containers by means of specifying stdcxx in makefile. The only trouble is using Genode allocators/new operator instead of native C++ ones.
I have this only in mind and even did not start to do anything. =)
Kind regards, Sergey.
Hi Sergey,
W.R.T. Genode allocators and STL - I would of thought that you would not need to do this directly, but that the Genode stdc++ port backs onto a port of malloc, that in turn backs onto Genode allocator calls.
Is there something specific you are trying to do?
Daniel
On 03/21/2013 12:30 AM, Norman Feske wrote:
Hello Sergey,
I am sorry for off-topic. Is it possible to use STL containers using mentioned way? (change target.mk)
yes. In order to use the C++ standard library, you will need to prepare it first. To do that, just issue the following command from within the 'libports' repository:
make prepare PKG=stdcxx
This will download and integrate the library with Genode. Also make sure that the 'libports' repository is specified in your build directory's 'REPOSITORIES' declaration (in '<build-dir>/etc/build.conf').
Please note, however, that we at Genode Labs don't use the C++ standard library much. So the stdcxx library does not receive much testing from our side. If you encounter problems with it, please bear with us. Also, some modern bits such as C++11 concurrency features are not supported. The containers should work without problems though.
And one more question: how can one use STL containers with Genode new placement operator?
I cannot help you here because I lack proficiency with STL. Maybe Daniel can help out here?
Best regards Norman
Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_mar
Genode-main mailing list Genode-main@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/genode-main
In theory, as Norman mentioned, you should be OK with STL on Genode. In this context you shouldn't use the Genode allocator/new operator. I have not tested them extensively either so I would be interested to hear how you get on.
Daniel
On 03/21/2013 11:34 AM, Sergey Grekhov wrote:
Hello, Daniel! Thanks for your answer!
My interest about STL containers has quite simple prerequisite. As far as I know Genode implementations of data structure (particularly, Avl_tree) has some issues and Genodelabs does not recommend it for usage in user applications. The reasons that issues are still not resolved is quite serious: Genode is kept as simple as possible. So, I was thinking about making a kind of porting of STL to Genode. But, it seems that this work is redundant since it is possible to use containers by means of specifying stdcxx in makefile. The only trouble is using Genode allocators/new operator instead of native C++ ones.
I have this only in mind and even did not start to do anything. =)
Kind regards, Sergey.
Hi Sergey,
W.R.T. Genode allocators and STL - I would of thought that you would not need to do this directly, but that the Genode stdc++ port backs onto a port of malloc, that in turn backs onto Genode allocator calls.
Is there something specific you are trying to do?
Daniel
On 03/21/2013 12:30 AM, Norman Feske wrote:
Hello Sergey,
I am sorry for off-topic. Is it possible to use STL containers using mentioned way? (change target.mk)
yes. In order to use the C++ standard library, you will need to prepare it first. To do that, just issue the following command from within the 'libports' repository:
make prepare PKG=stdcxx
This will download and integrate the library with Genode. Also make sure that the 'libports' repository is specified in your build directory's 'REPOSITORIES' declaration (in '<build-dir>/etc/build.conf').
Please note, however, that we at Genode Labs don't use the C++ standard library much. So the stdcxx library does not receive much testing from our side. If you encounter problems with it, please bear with us. Also, some modern bits such as C++11 concurrency features are not supported. The containers should work without problems though.
And one more question: how can one use STL containers with Genode new placement operator?
I cannot help you here because I lack proficiency with STL. Maybe Daniel can help out here?
Best regards Norman
Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_mar
Genode-main mailing list Genode-main@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/genode-main
Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_mar
Genode-main mailing list Genode-main@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/genode-main
Hello Sergey,
My interest about STL containers has quite simple prerequisite. As far as I know Genode implementations of data structure (particularly, Avl_tree) has some issues and Genodelabs does not recommend it for usage in user applications. The reasons that issues are still not resolved is quite serious: Genode is kept as simple as possible. So, I was thinking
with the term "issue", are you referring to bugs and unexpected behavior, or are you just finding those data structures limiting? If the former is the case, I would be very grateful if you shared details about the problems you encountered, so that we can remedy them.
The few data structures provided by Genode do merely exist to support the operation of the base framework. By rigidly following the microkernel construction principle, they comprise only those features that cannot be left out. This is crucial for keeping the complexity of the base system low. They should not be mistaken as an application framework.
For components where low complexity and tight control over memory allocations is critical (e.g., for resource multiplexers), I prefer to stick with the sparse facilities provided by Genode's base API. In contrast, for higher-level applications, the use of application-geared APIs (such as STL, Qt and the like) seems more appropriate.
about making a kind of porting of STL to Genode. But, it seems that this work is redundant since it is possible to use containers by means of specifying stdcxx in makefile. The only trouble is using Genode allocators/new operator instead of native C++ ones.
To integrate STL with Genode, you can define a new operator handing out anonymous memory as follows:
void *operator new (Genode::size_t size) { return Genode::env()->heap()->alloc(size); }
This operator will then be used by the standard C++ library and the STL containers. By doing that, you are of course sacrificing the ability to tightly guard allocations. For example, if your program is a server, it will not be able to use distinct heap partitions to keep track of the allocations performed on behalf of different clients. On the other hand, many components do not need such strict accounting. The decision between tight control (Genode API only) and convenience (Genode API + C++ standard library) is a trade-off and comes down to a judgment case by case.
Best regards Norman
Hello, Norman!
22.03.2013, 02:13, "Norman Feske" <norman.feske@...1...>:
Hello Sergey,
My interest about STL containers has quite simple prerequisite. As far as I know Genode implementations of data structure (particularly, Avl_tree) has some issues and Genodelabs does not recommend it for usage in user applications. The reasons that issues are still not resolved is quite serious: Genode is kept as simple as possible. So, I was thinking
with the term "issue", are you referring to bugs and unexpected behavior, or are you just finding those data structures limiting? If the former is the case, I would be very grateful if you shared details about the problems you encountered, so that we can remedy them.
Well, there are mainly two problems I know about tree: 1) it allows to change the key of the element which was already inserted and in that case no any re-balancing will be performed; if one add here the fact that each node in the tree has a link to its parent, there appears a theoretical possibility to get a cycle in the tree when searching for something 2) current implementation of AVL-tree is quite simple and it takes a lot of time to rebalance it; there are types of balanced trees (e.g. red-black tree) which has more cheap cost of operation "balance", but these types are more complex in terms of internal structures
The few data structures provided by Genode do merely exist to support the operation of the base framework. By rigidly following the microkernel construction principle, they comprise only those features that cannot be left out. This is crucial for keeping the complexity of the base system low. They should not be mistaken as an application framework.
For components where low complexity and tight control over memory allocations is critical (e.g., for resource multiplexers), I prefer to stick with the sparse facilities provided by Genode's base API. In contrast, for higher-level applications, the use of application-geared APIs (such as STL, Qt and the like) seems more appropriate.
about making a kind of porting of STL to Genode. But, it seems that this work is redundant since it is possible to use containers by means of specifying stdcxx in makefile. The only trouble is using Genode allocators/new operator instead of native C++ ones.
To integrate STL with Genode, you can define a new operator handing out anonymous memory as follows:
void *operator new (Genode::size_t size) { return Genode::env()->heap()->alloc(size); }
This operator will then be used by the standard C++ library and the STL containers. By doing that, you are of course sacrificing the ability to tightly guard allocations. For example, if your program is a server, it will not be able to use distinct heap partitions to keep track of the allocations performed on behalf of different clients. On the other hand, many components do not need such strict accounting. The decision between tight control (Genode API only) and convenience (Genode API + C++ standard library) is a trade-off and comes down to a judgment case by case.
Yes, this is a trade off which is not in spirit of Genode. At least env()->heap() is guarded by the ram_quota of the started app. I will think on the possibility to insert allocator in this scheme - that looks interesting. =)
Best regards Norman
-- Dr.-Ing. Norman Feske Genode Labs
http://www.genode-labs.com · http://genode.org
Genode Labs GmbH · Amtsgericht Dresden · HRB 28424 · Sitz Dresden Geschäftsführer: Dr.-Ing. Norman Feske, Christian Helmuth
Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_mar _______________________________________________ Genode-main mailing list Genode-main@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/genode-main
Hello Sergey,
Well, there are mainly two problems I know about tree:
- it allows to change the key of the element which was already
inserted and in that case no any re-balancing will be performed; if one add here the fact that each node in the tree has a link to its parent, there appears a theoretical possibility to get a cycle in the tree when searching for something
that is because there is no key stored in the nodes but the user of the 'Avl_tree' template supplies the 'higher' function as part of its policy. This function may evaluate any portion of the node data. If non-constant members of node are incorporated in the evaluation, the tree is prone to become inconsistent.
Do you think that this is a weakness of the class template? To me, it seems that the documentation should do a better job about educating users of the template about the meaning and restrictions of the 'higher' function. Of course, a way to prevent such misuse by design would be preferable, but I don't see any. Do you have an idea of how to achieve that?
The kind of "issue" looks somehow similar to how the 'List' class template is not thread safe. So it can be misused when accessed by multiple threads. Would that also qualify as a limitation? I don't think so.
- current implementation of AVL-tree is quite simple and it takes a
lot of time to rebalance it; there are types of balanced trees (e.g. red-black tree) which has more cheap cost of operation "balance", but these types are more complex in terms of internal structures
I am open to replace the current implementation by a faster one, given that there is evidence of the benefit, and that the new implementation is not unreasonably more complex. With evidence, I am referring to an application benchmark that clearly shows the benefit of using the new implementation in the base framework.
Yes, this is a trade off which is not in spirit of Genode. At least env()->heap() is guarded by the ram_quota of the started app. I will think on the possibility to insert allocator in this scheme - that looks interesting. =)
The hook of providing a custom anonymous new operator also allows you to explore the transparent use of alternative allocator implementations. For example, if 'env()->heap()' is too slow for you, the hook enables you to use a better allocator implementation that uses 'env()->heap()' merely to allocate chunks of backing store. So 'env()->heap()' is called rarely and no longer sits on the critical path.
For an example of such an allocator, you may take a look at our 'malloc' implementation at 'libports/src/lib/libc/malloc.cc'. This slab-based allocator is much more efficient and has less meta-data overhead than the AVL-tree-based best-fit allocator used by 'env()->heap()'. We introduced this allocator because it turned out to speed up the lwIP stack (which calls malloc/free multiple times per packet) by circa 20%.
Best regards Norman
Hello, Norman! I understand your position and it looks absolutely reasonable. Both issues are actually relate to trade-off between complexity and simplicity. I know that simplicity is one of your key points in design. That's why I thought that if someone needs specific data structures it would be nice to give ability to use STL. Besides, porting some generic code is fun. =)
Thanks for the hint about malloc - I will take a look!
Kind regards, Sergey.
Hello Sergey,
Well, there are mainly two problems I know about tree:
- it allows to change the key of the element which was already
inserted and in that case no any re-balancing will be performed; if one add here the fact that each node in the tree has a link to its parent, there appears a theoretical possibility to get a cycle in the tree when searching for something
that is because there is no key stored in the nodes but the user of the 'Avl_tree' template supplies the 'higher' function as part of its policy. This function may evaluate any portion of the node data. If non-constant members of node are incorporated in the evaluation, the tree is prone to become inconsistent.
Do you think that this is a weakness of the class template? To me, it seems that the documentation should do a better job about educating users of the template about the meaning and restrictions of the 'higher' function. Of course, a way to prevent such misuse by design would be preferable, but I don't see any. Do you have an idea of how to achieve that?
The kind of "issue" looks somehow similar to how the 'List' class template is not thread safe. So it can be misused when accessed by multiple threads. Would that also qualify as a limitation? I don't think so.
- current implementation of AVL-tree is quite simple and it takes a
lot of time to rebalance it; there are types of balanced trees (e.g. red-black tree) which has more cheap cost of operation "balance", but these types are more complex in terms of internal structures
I am open to replace the current implementation by a faster one, given that there is evidence of the benefit, and that the new implementation is not unreasonably more complex. With evidence, I am referring to an application benchmark that clearly shows the benefit of using the new implementation in the base framework.
Yes, this is a trade off which is not in spirit of Genode. At least env()->heap() is guarded by the ram_quota of the started app. I will think on the possibility to insert allocator in this scheme - that looks interesting. =)
The hook of providing a custom anonymous new operator also allows you to explore the transparent use of alternative allocator implementations. For example, if 'env()->heap()' is too slow for you, the hook enables you to use a better allocator implementation that uses 'env()->heap()' merely to allocate chunks of backing store. So 'env()->heap()' is called rarely and no longer sits on the critical path.
For an example of such an allocator, you may take a look at our 'malloc' implementation at 'libports/src/lib/libc/malloc.cc'. This slab-based allocator is much more efficient and has less meta-data overhead than the AVL-tree-based best-fit allocator used by 'env()->heap()'. We introduced this allocator because it turned out to speed up the lwIP stack (which calls malloc/free multiple times per packet) by circa 20%.
Best regards Norman
-
Daniel Waddington -
Norman Feske -
Sergey Grekhov