Thanks for the clarification Josef. But it turns out the application had a dependency specifically on <cryptopp/*>. So I went ahead and ported it. Barring some test files which were thrown undefined references, I was able to build the library. So far, so good.

But, here is the problem. I filled the directories of cryptopp include, and the new library (it's called ndn-cxx) I'm porting has its dependency in this crypto++ lib. I've added it to the LIBS of ndn-cxx. One of the files called random.cpp #includes a list of <cryptopp/*> header files, which defines a set of SeededRandom number classes. For some reason or the other, it throws an error saying "'SeededRandom number' in namespace CryptoPP does not name a type", even though it is present in the header file.

A similar error happens when I try to compile the files without specifying the $(notdir ...) option in the .mk file of lib/mk of ndn-cxx. Any ideas will help, because I'm very nearly done with the port and I'm out of ideas. Sorry if it seems like a bit of a rant.

Thanks and regards Aditya

On Wed, May 6, 2015 at 1:10 AM, Josef Söntgen < josef.soentgen@...1...> wrote:

Hello Aditya,

• Aditya Kousik <adit267.kousik@...9...> [2015-05-05 11:45:29 -0700]:

...

To use libcrypto++, I found online that openssl will suffice. But while there is a port file for both libraries, there's no openssl.mk in

lib/mk.

...

Grepping the mk files I saw a line in libcrypto.inc which calls select_from_ports,openssl. So, is it the case that we prepare_port for openssl but add LIBS += libcrypto in our target.mk files?

Yes, libcrypto as well as libssl are part of OpenSSL, so you have to prepare it when you want to use one of these libraries.

Regards Josef

---

One dashboard for servers and applications across Physical-Virtual-Cloud Widest out-of-the-box monitoring support with 50+ applications Performance metrics, stats and reports that give you Actionable Insights Deep dive visibility with transaction tracing using APM Insight. http://ad.doubleclick.net/ddm/clk/290420510;117567292;y _______________________________________________ genode-main mailing list genode-main@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/genode-main

Hello Christian,

Here's a brief summary of what I've done. The project is hosted at github.com/named-data/ndn-cxx. It's a novel Internet architecture that is becoming quite popular.

I wrote the port file to clone the project to the contrib folder and copy all .h files in the ndn-cxx/src folder, manually adding all folders.

Next I wrote a target.mk file in test/libports/ndn-cxx with a dummy target.mk file that adds the ndn-cxx library as LIBS.

ndn-cxx by itself has dependencies on boost libraries, libcrypto++, libsqlite3. Sqlite3 was already present. I wrote a libboost.port and cryptopp.port to compile before ndn-cxx, while manually adding the required header files of boost.

Here's the commit of all changes I've done so far. https://github.com/adikou/genode/commit/df5f4cd7730095ad1dea42151946df63baf1...

This builds libc, libm, libc_lxip, zlib, libbz2 and additionally cryptopp, boost.

I assumed since all header files concerned were put in the contrib/crytopp-<hash>/include folder, they will be included during compilation. I've added a line in lib/import/import-ndn-cxx.mk to add INC_DIR as well.

I hope that establishes some ground on what I'm attempting to doHere's a brief summary of what I've done. The project is hosted at github.com/named-data/ndn-cxx.git. It's a novel Internet architecture that is becoming quite popular.

I wrote the port file to clone the project to the contrib folder and copy all .h files in the ndn-cxx/src folder, manually adding all folders.

Next I wrote a target.mk file in test/libports/ndn-cxx with a dummy target.mk file that adds the ndn-cxx library as LIBS.

ndn-cxx by itself has dependencies on boost libraries, libcrypto++, libsqlite3. Sqlite3 was already present. I wrote a libboost.port and cryptopp.port to compile before ndn-cxx, while manually adding the required header files of boost.

Here's the commit of all changes I've done so far. https://github.com/adikou/genode/commit/df5f4cd7730095ad1dea42151946df63baf1...

This builds libc, libm, libc_lxip, zlib, libbz2 and additionally cryptopp, boost.

I assumed since all header files concerned were put in the contrib/crytopp-<hash>/include folder, they will be included during compilation. I've added a line in lib/import/import-ndn-cxx.mk to add INC_DIR as well.

Thanks, Aditya

On Wed, May 6, 2015 at 3:13 AM, Christian Helmuth < christian.helmuth@...1...> wrote:

Hello Aditya,

On Wed, May 06, 2015 at 01:52:32AM -0700, Aditya Kousik wrote:

...

For some reason or the other, it throws an error saying "'SeededRandom number' in namespace CryptoPP does not name a type", even though it is present in the header file.

To be honest, I see no chance to help you identifying the cause of this compilation error without the sources or at least the relevant parts of your build log. Does the compiler also warn about other stuff like missing include files?

Regards

Christian Helmuth Genode Labs

http://www.genode-labs.com/ · http://genode.org/ https://twitter.com/GenodeLabs · /ˈdʒiː.nəʊd/

Genode Labs GmbH · Amtsgericht Dresden · HRB 28424 · Sitz Dresden Geschäftsführer: Dr.-Ing. Norman Feske, Christian Helmuth

---

One dashboard for servers and applications across Physical-Virtual-Cloud Widest out-of-the-box monitoring support with 50+ applications Performance metrics, stats and reports that give you Actionable Insights Deep dive visibility with transaction tracing using APM Insight. http://ad.doubleclick.net/ddm/clk/290420510;117567292;y _______________________________________________ genode-main mailing list genode-main@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/genode-main

Hello Christian,

That's true. I hadn't committed the mk files for ndn-cxx. I have now. The repository still won't build because there's a WAF based config file - ndn-cxx-config.hpp - that needs to be generated. This file is simply a header file with #defines, generated in build/src by running ./waf configure and ./waf. I just copied this file and the rest of the build worked.... Till it hit the roadblock.

The branch is at github.com/adikou/genode.git.

Thank you for the consideration. On May 6, 2015 8:19 AM, "Christian Helmuth" < christian.helmuth@...1...> wrote:

Hello Aditya,

I fetched your branch, prepared the added ports, and built test/libports/cryptopp and test/libports/boost successfully. What your branch is missing is the incomplete/non-working library definition for ndn-cxx.

Greets

Christian Helmuth Genode Labs

http://www.genode-labs.com/ · http://genode.org/ https://twitter.com/GenodeLabs · /ˈdʒiː.nəʊd/

Genode Labs GmbH · Amtsgericht Dresden · HRB 28424 · Sitz Dresden Geschäftsführer: Dr.-Ing. Norman Feske, Christian Helmuth

---

One dashboard for servers and applications across Physical-Virtual-Cloud Widest out-of-the-box monitoring support with 50+ applications Performance metrics, stats and reports that give you Actionable Insights Deep dive visibility with transaction tracing using APM Insight. http://ad.doubleclick.net/ddm/clk/290420510;117567292;y _______________________________________________ genode-main mailing list genode-main@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/genode-main

It's pretty straightforward. Just Replace the line #include "ndn-cxx-config.hpp" in contrib/ndn-cxx-<hash>/src/lib/ndn-cxx/src/common.hpp with

#define NDN_CXX_HAVE_IS_DEFAULT_CONSTRUCTIBLE 1 #define NDN_CXX_HAVE_IS_MOVE_CONSTRUCTIBLE 1 #define NDN_CXX_HAVE_IS_MOVE_ASSIGNABLE 1 #define NDN_CXX_HAVE_CXX_FRIEND_TYPENAME 1 #define NDN_CXX_HAVE_CXX_OVERRIDE_FINAL 1 #define NDN_CXX_HAVE_PTHREAD 1 #define NDN_CXX_HAVE_RT 1 #define NDN_CXX_HAVE_GETPASS 1 #define NDN_CXX_HAVE_RTNETLINK 1 #define NDN_CXX_HAVE_SQLITE3 1 #define NDN_CXX_HAVE_CRYPTOPP_CONFIG_H 1

or use these values for the ndn-cxx-config.hpp itself. Either method works.

I'm really grateful for your help. Thank you!

Aditya.

On Wed, May 6, 2015 at 9:04 AM, Christian Helmuth < christian.helmuth@...1...> wrote:

Hello Aditya,

On Wed, May 06, 2015 at 08:46:44AM -0700, Aditya Kousik wrote:

...

That's true. I hadn't committed the mk files for ndn-cxx. I have now. The repository still won't build because there's a WAF based config file - ndn-cxx-config.hpp - that needs to be generated. This file is simply a header file with #defines, generated in build/src by running ./waf configure and ./waf.

Unfortunately, there is no waf package available for Ubuntu 12.04, which is the Linux distro I'm using currently. Also, my spare time is very limited, so I'm unable to find an alternative solution to get waf.

So, could you please add the file to your branch or post it to the list?

Regards

Christian Helmuth Genode Labs

http://www.genode-labs.com/ · http://genode.org/ https://twitter.com/GenodeLabs · /ˈdʒiː.nəʊd/

Genode Labs GmbH · Amtsgericht Dresden · HRB 28424 · Sitz Dresden Geschäftsführer: Dr.-Ing. Norman Feske, Christian Helmuth

---

One dashboard for servers and applications across Physical-Virtual-Cloud Widest out-of-the-box monitoring support with 50+ applications Performance metrics, stats and reports that give you Actionable Insights Deep dive visibility with transaction tracing using APM Insight. http://ad.doubleclick.net/ddm/clk/290420510;117567292;y _______________________________________________ genode-main mailing list genode-main@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/genode-main

It's a weird scenario that I had the exact same epiphany, at about the same time. I found that all cryptopp/* files have such #defs. Primarily done in cryptopp/config.h, there's a #if defined(__unix__) then #define OS_RNG_AVAILABLE, and so on.

So, you may need to contact GCC and ask them to add __genode__ for the defined preprocessor. But, with your help, I've discovered a way to proceed. Thank you very much, I'll keep you updated.

Cheers, Aditya

On Thu, May 7, 2015 at 12:06 AM, Christian Helmuth < christian.helmuth@...1...> wrote:

Hello Aditya,

from what I discovered I fear you have to do some additional porting work as the building produces several errors. Please try "make --keep-going ..." to see them all and get about 2000 lines of warnings and errors (mostly from cryptopp). The most suspicious messages from ndn-cxx are

/plain/krishna/src/genode/genode.git/contrib/ndn-cxx-d868c173a4ad62c998b4be62fe3b9dca91e42160/src/lib/ndn-cxx/src/security/sec-tpm-file.cpp: In member function ‘virtual void ndn::SecTpmFile::generateKeyPairInTpm(const ndn::Name&, const ndn::KeyParams&)’: /plain/krishna/src/genode/genode.git/contrib/ndn-cxx-d868c173a4ad62c998b4be62fe3b9dca91e42160/src/lib/ndn-cxx/src/security/sec-tpm-file.cpp:131:4: error: ‘AutoSeededRandomPool’ was not declared in this scope

In file included from /plain/krishna/src/genode/genode.git/contrib/ndn-cxx-d868c173a4ad62c998b4be62fe3b9dca91e42160/src/lib/ndn-cxx/src/security/sec-tpm-osx.cpp:24:0: /plain/krishna/src/genode/genode.git/contrib/ndn-cxx-d868c173a4ad62c998b4be62fe3b9dca91e42160/src/lib/ndn-cxx/src/security/sec-tpm-osx.hpp:30:2: error: #error "This files should not be compiled ..." /plain/krishna/src/genode/genode.git/contrib/ndn-cxx-d868c173a4ad62c998b4be62fe3b9dca91e42160/src/lib/ndn-cxx/src/security/sec-tpm-osx.cpp:38:43: fatal error: CoreFoundation/CoreFoundation.h: No such file or directory

For the first issue please have a look at the following lines in cryptopp/osrng.h

8 #ifdef OS_RNG_AVAILABLE ... 155 #endif

It seems the define is not set in your case and so the contents of the header are not includes in the build. To find this kind of problems I suggest to modify the compilation of the broken source file to just do the preprocessor step (compiler flag -E) and analyze the result. In your case I was looking for AutoSeededRandomPool, which was not there.

Good luck

Christian Helmuth Genode Labs

http://www.genode-labs.com/ · http://genode.org/ https://twitter.com/GenodeLabs · /ˈdʒiː.nəʊd/

Genode Labs GmbH · Amtsgericht Dresden · HRB 28424 · Sitz Dresden Geschäftsführer: Dr.-Ing. Norman Feske, Christian Helmuth

---

One dashboard for servers and applications across Physical-Virtual-Cloud Widest out-of-the-box monitoring support with 50+ applications Performance metrics, stats and reports that give you Actionable Insights Deep dive visibility with transaction tracing using APM Insight. http://ad.doubleclick.net/ddm/clk/290420510;117567292;y _______________________________________________ genode-main mailing list genode-main@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/genode-main

Hello Aditya,

* Aditya Kousik <adit267.kousik@...9...> [2015-05-07 17:08:00 -0700]:

2. Compilation of ndn-cxx goes fine (after adding ndn-cxx-config.hpp).

The final set of errors are undefined references. I'm facing similar problems to the article written about porting SDL_net. undefined references to 'kqueue', 'kevent' and 'sendmsg'.

I've found a few implementations of kqueue(), kevent() in the original libc repository : they're defined in kern_event.c which is not present in the current libc port. Same for sendmsg. I've been trying to add kevent and kqueue by changing the libc port by adding sys/kern to the list of exports and adding a mk file for the libc-kern component. I'm wondering if I should go to the extent of doing this because just by looking at the code I can see that there will be a cascade of dependencies within sys/kern itself that I now have to deal with. Suggestions would help - I'm kind of lost with this exact decision that was mentioned in porting SDL_net.

That is the kernel side of FreeBSD's kqueue(2) implementation and is not part of the libc. If you take a closer look at our libc you will see that we actually implement the system call functions, i.e., we provide the functionality normally expected from the kernel. Therefore, your job would be to write the implementation of the kernel side of kqueue(2) like it is done for, e.g., select(2) or rather emulate its behaviour.

The weirdest part in this, is that I can't seem to find the place in the ndn-cxx code that even makes a call or even includes sys/event.h. I grepped the words 'kqueue', 'kevent' and 'sendmsg' and they're present in the object files but nowhere to be found in the source files!

Yeah, I bet boost uses kqueue and the like in its asynchronous I/O backend because it thinks it is running on FreeBSD. Hence, you could try to configure boost in such a way that it uses poll(2) or select(2) (that would be the easy way) or you could extend the libc (the not so easy way). Since you would not gain much performance-wise (*), I would take the easy way.

Part 2: Since the contrib/library-<hash>/ folder has both include/ as well as src/ folder, which one actually becomes the INC_DIR in .mk file? Because whatever changes I made when I set the include/ folder as INC_DIR were not getting reflected during compilation. So, there's a side query about that.

It depends on the include search path and how the headers are included. If the header files are included locally, by using “#include "..."” that is, the compiler might take them over the ones specified in INC_DIR.

So, the import .mk always should use the include/ directory. The lib or target .mk should also always use the include/ directory but you might have to overlay certain header files so that local ones in the src/ directory are not picked up if you make changes to them. Ideally, changes to any files should be made _before_ they are copied to the include/ directory when preparing the port.

Regards Josef

(*) In our case, it really depends on how the backend is implemented rather than which API is used.

Hello Josef,

The porting task is challenging, but interesting. For some reason or the other, during preprocessing, boost kept configuring its files into BSD. This happens because one of #if defined(__FreeBSD__) || defined(__NetBSD__) || defined(__OpenBSD__) || defined(__DragonFly__) is throwing TRUE, so the boost configuration follows BSD and hence, was including kqueue.

Fortunately for me, the hierarchy of config was convenient. There's one reactor.hpp which includes either epoll_reactor, or kqueue_reactor, or dev_poll_reactor, or most importantly - the default was select_reactor.hpp which uses the select call. I was able to fix this by hacking at the config files.

Which brings me to a general question: How is the build system isolated from the host configurations? How are the preprocess primitives supposed to respond? Can we include a genode-sepcific configuration file to help build in such cases - to direct output of such definitions.

The only undefined reference I'm facing now, is to sendmsg(), which I'm assuming is a similar misconfiguration issue.

Cheers, Aditya

On Fri, May 8, 2015 at 2:50 AM, Josef Söntgen < josef.soentgen@...1...> wrote:

Hello Aditya,

• Aditya Kousik <adit267.kousik@...9...> [2015-05-07 17:08:00 -0700]:

...

2. Compilation of ndn-cxx goes fine (after adding ndn-cxx-config.hpp).

The final set of errors are undefined references. I'm facing similar problems to the article written about porting SDL_net. undefined references to 'kqueue', 'kevent' and 'sendmsg'.

I've found a few implementations of kqueue(), kevent() in the original

libc

...

repository : they're defined in kern_event.c which is not present in the current libc port. Same for sendmsg. I've been trying to add kevent and kqueue by changing the libc port by adding sys/kern to the list of

exports

...

and adding a mk file for the libc-kern component. I'm wondering if I

should

...

go to the extent of doing this because just by looking at the code I can see that there will be a cascade of dependencies within sys/kern itself that I now have to deal with. Suggestions would help - I'm kind of lost with this exact decision that was mentioned in porting SDL_net.

That is the kernel side of FreeBSD's kqueue(2) implementation and is not part of the libc. If you take a closer look at our libc you will see that we actually implement the system call functions, i.e., we provide the functionality normally expected from the kernel. Therefore, your job would be to write the implementation of the kernel side of kqueue(2) like it is done for, e.g., select(2) or rather emulate its behaviour.

...

The weirdest part in this, is that I can't seem to find the place in the ndn-cxx code that even makes a call or even includes sys/event.h. I

grepped

...

the words 'kqueue', 'kevent' and 'sendmsg' and they're present in the object files but nowhere to be found in the source files!

Yeah, I bet boost uses kqueue and the like in its asynchronous I/O backend because it thinks it is running on FreeBSD. Hence, you could try to configure boost in such a way that it uses poll(2) or select(2) (that would be the easy way) or you could extend the libc (the not so easy way). Since you would not gain much performance-wise (*), I would take the easy way.

...

Part 2: Since the contrib/library-<hash>/ folder has both include/ as

well

...

as src/ folder, which one actually becomes the INC_DIR in .mk file?

Because

...

whatever changes I made when I set the include/ folder as INC_DIR were

not

...

getting reflected during compilation. So, there's a side query about

that.

It depends on the include search path and how the headers are included. If the header files are included locally, by using “#include "..."” that is, the compiler might take them over the ones specified in INC_DIR.

So, the import .mk always should use the include/ directory. The lib or target .mk should also always use the include/ directory but you might have to overlay certain header files so that local ones in the src/ directory are not picked up if you make changes to them. Ideally, changes to any files should be made _before_ they are copied to the include/ directory when preparing the port.

Regards Josef

(*) In our case, it really depends on how the backend is implemented rather than which API is used.

---

One dashboard for servers and applications across Physical-Virtual-Cloud Widest out-of-the-box monitoring support with 50+ applications Performance metrics, stats and reports that give you Actionable Insights Deep dive visibility with transaction tracing using APM Insight. http://ad.doubleclick.net/ddm/clk/290420510;117567292;y _______________________________________________ genode-main mailing list genode-main@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/genode-main

On Wed, May 06, 2015 at 06:18:28AM -0700, Aditya Kousik wrote:

ndn-cxx by itself has dependencies on boost libraries, libcrypto++, libsqlite3. Sqlite3 was already present. I wrote a libboost.port and cryptopp.port to compile before ndn-cxx, while manually adding the required header files of boost.

Aditya,

Use the sqlite port with some caution, I haven't run it against the set of tests that upstream uses and I haven't used it on any long-term databases. If you think its causing you problems please let me know.

Good luck with the port, Emery