Translation Table Memory Attribute bits
Bob Stewart
robjsstewart at ...196...
Sun Sep 7 19:09:37 CEST 2014
Stefan,
I believe I see where the issue is and it relates to another change
I need to make to released kernel/core code to make Genode run on an
Am335x-based Beaglebone board.
Here are all the changes I need to make to a release since 13.08:
1. The Beaglebone board that I am using uses the TL16C750 UART chip. The
Pandaboard also uses this chip and code exists in base-hw to provide
serial line support. This code is hard-coded to use uart3 as the console
port which is fine for the Pandaboard. The Beagleboard used uart0, so I
alter serial.h accordingly.
2. As I've mentioned in this email thread, without modify any of the
translation table memory access bits, the kernel will never initialize.
Once the MMU is turned on no further output or apparent cpu activity
occurs. This is fixed if the access bits for device memory is set to
Tex[0:2] = 0, C = 0, and B =1. As you will notice from the TI code
snippet below, device memory is shared and these bits setting are in
agreement with the Arm v7 spec as defined in table B3-10. I also change
the access control bits to make cacheability agree with that used by the
TI code. After those changes everything works fine and the kernel
initializes and user programs run ok, until release 14.08
3. The Am335x requires a kernel privilege level to write to to sets of
registers that are critical for any embedded application. The first set
of registers are in the "Control Module" where register settings
enable/disable subsystems and set up various functional clocks. The
second set of registers are the pin-mux registers through which the
operating modes available and gpio pins can be set. These registers
cannot be written to from userland. So I created two kernel calls to
handle writing to each of these register sets. That worked fine, until
14.08.
So back to the current issue. Currently the kernel initializes and init
starts and the various modules specified in the run script get executed
in threads. All the modules start up, but shortly after starting the
whole system appears to freeze. When I saw the MMU exceptions from my
debugging and they mostly related to the section entry for address
0x81000000, I assumed it was an issue with the bits in the translation
table entry. I was also confused by the output which shows a series of
MMU exceptions in sequence on that same section table entry. I still
don't understand why this occurs when the currently used translation
table entry is still valid. For example, the following debug output
sequence seems to me to indicated that each thread causes a MMU fault
even though the section entry in the translation table should still be
valid:
void Kernel::Thread::_mmu_exception(): f_addr 0x81000000 f_writes 0x0
f_pd 0x8143d088 f_signal 0x86 label timer
void Kernel::Thread::_mmu_exception(): f_addr 0x81000000 f_writes 0x0
f_pd 0x81474088 f_signal 0x88 label gpio_drv
void Kernel::Thread::_mmu_exception(): f_addr 0x81000000 f_writes 0x0
f_pd 0x814a9088 f_signal 0x8a label ctl_module_drv
void Kernel::Thread::_mmu_exception(): f_addr 0x81000000 f_writes 0x0
f_pd 0x814e0088 f_signal 0x8c label bbb_heart_beat_led
void Kernel::Thread::_mmu_exception(): f_addr 0x81043b40 f_writes 0x1
f_pd 0x8143d088 f_signal 0x86 label timer
void Kernel::Thread::_mmu_exception(): f_addr 0x8104ab50 f_writes 0x1
f_pd 0x81474088 f_signal 0x88 label gpio_drv
void Kernel::Thread::_mmu_exception(): f_addr 0x81049310 f_writes 0x1
f_pd 0x814a9088 f_signal 0x8a label ctl_module_drv
void Kernel::Thread::_mmu_exception(): f_addr 0x8103ada0 f_writes 0x1
f_pd 0x814e0088 f_signal 0x8c label bbb_heart_beat_led
What is it that I'm missing?
The real problem is being caused due to the change to the released code
(for 14.05 or 14.08?) for kernel calls which are now split into two
tables, for kernel calls that anyone can make, and kernel calls that
only core can make. I made what I thought were the appropriate changes
to allow my two kernel calls to execute from userland, but, the Control
Module register writes appear to be hanging the system. I'm currently
investigating the reason for the hang.
Thanks,
Bob
On 09/05/2014 01:33 PM, Bob Stewart wrote:
> Hi Stefan,
> The TI Sitara Family of processors Starterware contains code
> setting up the translation tables... here's the relevant snippet--
>
> /*
> ** Function to setup MMU. This function Maps three regions ( 1. DDR
> ** 2. OCMC and 3. Device memory) and enables MMU.
> */
> void MMUConfigAndEnable(void)
> {
> /*
> ** Define DDR memory region of AM335x. DDR can be configured as Normal
> ** memory with R/W access in user/privileged modes. The cache attributes
> ** specified here are,
> ** Inner - Write through, No Write Allocate
> ** Outer - Write Back, Write Allocate
> */
> REGION regionDdr = {
> MMU_PGTYPE_SECTION, START_ADDR_DDR, NUM_SECTIONS_DDR,
> MMU_MEMTYPE_NORMAL_NON_SHAREABLE(MMU_CACHE_WT_NOWA,
> MMU_CACHE_WB_WA),
> MMU_REGION_NON_SECURE, MMU_AP_PRV_RW_USR_RW,
> (unsigned int*)pageTable
> };
> /*
> ** Define OCMC RAM region of AM335x. Same Attributes of DDR region given.
> */
> REGION regionOcmc = {
> MMU_PGTYPE_SECTION, START_ADDR_OCMC, NUM_SECTIONS_OCMC,
> MMU_MEMTYPE_NORMAL_NON_SHAREABLE(MMU_CACHE_WT_NOWA,
> MMU_CACHE_WB_WA),
> MMU_REGION_NON_SECURE, MMU_AP_PRV_RW_USR_RW,
> (unsigned int*)pageTable
> };
>
> /*
> ** Define Device Memory Region. The region between OCMC and DDR is
> ** configured as device memory, with R/W access in user/privileged modes.
> ** Also, the region is marked 'Execute Never'.
> */
> REGION regionDev = {
> MMU_PGTYPE_SECTION, START_ADDR_DEV, NUM_SECTIONS_DEV,
> MMU_MEMTYPE_DEVICE_SHAREABLE,
> MMU_REGION_NON_SECURE,
> MMU_AP_PRV_RW_USR_RW | MMU_SECTION_EXEC_NEVER,
> (unsigned int*)pageTable
> };
>
>
> I had to change the memory attributes for device memory to get the
> kernel to complete initialization. If B is not set the kernel silently
> page faults after the mmu is turned on.
>
> The .text address of 0x81000000 come from readelf.
>
> Yes I added a printk to get at the mmu exception.
>
> I can see one page fault but it appears to be a page fault on every
> access to the section entry for 0x81000000.
>
> Bob
>
>
>
> On 09/05/2014 08:30 AM, Stefan Kalkowski wrote:
>> Hi Bob,
>>
>> On 09/05/2014 01:43 PM, Bob Stewart wrote:
>>> Thanks for the reply Stefan.
>>>
>>> For the AM335x processors, the cacheability is required to be set to:
>>>
>>> Inner: Write thru, no write allocate
>>> Outer: Write back, write allocate.
>> Are you sure? Can you point me to the relevant information if it's
>> openly available? In the Cortex A8 and AM335x TRM I couldn't find that
>> restriction. I would assume that you don't have to change any memory
>> attributes, as we support Cortex A8 already (although this was
>> error-prone in the past).
>>
>>> So, in 14 .02 I set the Tex, C, and B bits accordingly and that worked
>>> fine. I just transferred those setting to 14.08.
>>>
>>> With the rework in that tlb area of the kernel for multi-processor
>>> support in 14.05 and 14.08, I assumed I was screwing something up in the
>>> translation table entry attribute bits.
>>>
>>> According to the ROM fs dump "Rom: [8113b000,8117aa24) init". it is the
>>> .text section of the program image that starts at 0x81000000.
>> Don't mix things up: the ROM fs dump gives you the physical memory where
>> the boot modules reside in. The program image layout can be seen by
>> e.g.: objdump, or readelf. In this example they are
>> coincidentally nearby.
>>
>> After re-reading your log output (I assume you've added the "void
>> Kernel::Thread::_mmu_exception()" printings?) it seems to me like the
>> init process produces a first page-fault, when executing its first
>> instruction, which is normal, and afterwards continues to run - so the
>> pagefault gets actually resolved, right? After that it gets additional
>> page-faults corresponding to its program flow. So what is the actual
>> weird behaviour? I can't see it given your output.
>>
>> Regards
>> Stefan
>>
>>> I'll dump the contents of the DFSR and see what that tells me later
>>> today. I'll also try run/printf as the program image, but the program
>>> image I'm using runs fine when built on 14.02.
>>>
>>> Thanks for the suggestion.
>>>
>>> Bob
>>>
>>> On 09/05/2014 03:36 AM, Stefan Kalkowski wrote:
>>>> Hi Bob,
>>>>
>>>> On 09/04/2014 03:33 PM, Bob Stewart wrote:
>>>>> I've never been able to get 14.05 or 14.08 working on my AM335X
>>>>> processor, which is not a big deal as 14.02 has everything I need for
>>>>> the applications I'm using that processor for. But out of curiosity:
>>>>>
>>>>> The issue on 14.08 may revolve around memory access bit rights in TLB
>>>>> table entries. To get 14.08 to initialize the kernel properly the memory
>>>>> access bits have to be set as Tex = 0, B = 1, and C = 0. These setting
>>>>> seem correct for a shared Device according to the Arm v7 ref manual.
>>>>> With those settings in place, the kernel initializes properly and
>>>>> eventually init runs. During the kernel initialization process Core_pd
>>>>> is called and translations are created for the program image (which
>>>>> starts at 0x81000000) and the core-only io memory regions. The
>>>>> translation table entries for the program image are of section size plus
>>>>> a small page so two entries are inserted in the translation table. The
>>>>> access bits and permission bits for the section entry are correct with
>>>>> the possible exception of the C bit, which in 14.08 appears never to be
>>>>> set and I wondered why that was, when it is used in 14.02.
>>>> We reworked a lot regarding ARM caches, shareability etc. within the
>>>> last months. Nowadays (release 14.08), on all Arm v7 platforms,
>>>> including Cortex A8, we set the following memory region attributes for
>>>> normal memory (!not device memory): Tex=0b101, C=0, B=1
>>>> That means: normal, inner- and outer-cacheable memory, with
>>>> write-back,write-allocate caching policy. Which works properly on all
>>>> our Cortex A8, Cortex A9, and Cortex A15 platforms.
>>>>
>>>>> Once the init thread runs, any reference to the translation table entry
>>>>> for the program image, the section entry mentioned above cause a mmu
>>>>> exception as the following partial debug output shows:
>>>>> ...
>>>>> start thread 3 'entrypoint' in program 1 'core' on processor 0/1
>>>>> start thread 4 'signal' in program 1 'core' on processor 0/1
>>>>> start thread 5 'pager_activation' in program 1 'core' on processor 0/1
>>>>> int main(): --- start init ---
>>>>> int main(): transferred 507 MB to init
>>>>> start thread 6 'init' in program 1 'core' on processor 0/1
>>>>> thread id is 0x7
>>>>> start thread 7 'init' in program 2 'init' on processor 0/1
>>>>> void Kernel::Thread::_mmu_exception(): f_addr 0x81000000 f_writes 0x0
>>>>> f_pd 0x813ed088 f_signal 0x7f
>>>>> label init
>>>>> int main(): --- init created, waiting for exit condition ---
>>>>> void Kernel::Thread::_mmu_exception(): f_addr 0x81045f60 f_writes 0x1
>>>>> f_pd 0x813ed088 f_signal 0x7f
>>>>> label init
>>>>> void Kernel::Thread::_mmu_exception(): f_addr 0x8102dab8 f_writes 0x0
>>>>> f_pd 0x813ed088 f_signal 0x7f
>>>>> label init
>>>>> ...
>>>>>
>>>>> Setting the C bit as it was set in 14.02 makes no difference, which I
>>>>> thought it would and it should be affecting caching behavior.
>>>>>
>>>>> Any thoughts on this behavoir?
>>>> Before thinking about a caching issue, I would investigate whether there
>>>> is no other issue. Above output shows that the whole kernel/core are
>>>> fully initialized and the init process is started. When the init process
>>>> tries to do some "write" operations it fails, right? So there is no
>>>> problem with the core's translation tables (Core_pd) at all.
>>>>
>>>> First of all, you should identify which kind of MMU exception was
>>>> triggered by the init process. Therefore, print out the DFSR (data fault
>>>> status register) directly after the corresponding faults occur. Does the
>>>> init binary also start at 0x81000000?
>>>>
>>>> Regards
>>>> Stefan
>>>>
>>>>> Thanks,
>>>>> Bob
>>>>>
>>>>> ------------------------------------------------------------------------------
>>>>> Slashdot TV.
>>>>> Video for Nerds. Stuff that matters.
>>>>> http://tv.slashdot.org/
>>>>> _______________________________________________
>>>>> genode-main mailing list
>>>>> genode-main at lists.sourceforge.net
>>>>> https://lists.sourceforge.net/lists/listinfo/genode-main
>>>>>
>>> ------------------------------------------------------------------------------
>>> Slashdot TV.
>>> Video for Nerds. Stuff that matters.
>>> http://tv.slashdot.org/
>>> _______________________________________________
>>> genode-main mailing list
>>> genode-main at lists.sourceforge.net
>>> https://lists.sourceforge.net/lists/listinfo/genode-main
>>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.genode.org/pipermail/users/attachments/20140907/5aa73a27/attachment.html>
More information about the users
mailing list