Understanding VirtualSetAttributes


Posted by Kurt Kennett 


Virtual Memory is fantastic. It allows you to create this personalized ‘view’ of the memory space of a computer, and rearrange where things are physically to suit your desires.  This is especially good for the organization of registers of memory-mapped peripherals.  By allocating a range of virtual memory, and mapping that memory onto a physical register bank, you get a pointer to a C-like structure containing fields that correspond to each register.



picture 


This procedure is fairly straightforward, and allows easy access to the hardware registers of many types of CPUs and peripherals in a Windows CE system.  Usually this is accomplished via one of two mechanisms:


1)      VirtualAlloc() + VirtualCopy() / VirtualFree()


a.       A range of virtual address space is allocated that can map the physical space needed.


b.      The virtual to physical mapping is set up for those pages


c.       When necessary, the range of virtual pages is released and the mapping destroyed.


-OR-


2)      MmMapIoSpace() / MmUnmapIoSpace()


a.       This effectively does both of the (a) and (b) steps above in one function call, with fixups on the address if it is not page-aligned.


The functions in (1) are exported to all drivers and applications.  The functions in (2) are implemented as part of the CEDDK (CE Driver Development Kit), and are customizable for any particular platform.


With some new processor systems, it becomes optional and sometimes necessary to set ‘attributes’ on virtual memory pages that map specific portions of physical memory. This is done to set qualities on the physical resources that are mapped.  For example, TLB (Translation Lookaside Buffer) entries for a set of virtually mapped pages may need to be configured with specific attributes in order to allow a particular bus on the system to access the memory. If an operating system has segmented data and code areas, it can set the ‘no execute’ bit to make sure the CPU doesn’t fetch instructions from data regions. Another example is ARM’s TrustZone technology, which uses a ‘secure’ bit to establish privilege on certain memory regions. One of the most common examples is a display buffer, where you do not want to cache writes to a frame buffer, but want to use a hardware write buffer instead.


To allow this type of customization, the VirtualSetAttributes() function is provided.  This allows us to manually call the function after the sequence in (1) above, or automatically in the function provided by an OEM in (2).  Using the mechanism in (2) is more desirable, as it leads to more portable code (less specificity with regard to setting attributes for an end system in driver code).  However, using (2) does not easily allow for multiple register banks to be allocated in one virtual page (leading to waste of virtual address space).  Depending on your needs, you can use whichever method you desire.


By default the memory system is generic and does not know which buses are used to access a particular piece of physical memory.  Because of this, if the default flags are used, a peripheral bus might not be accessed properly.  You can set the attributes properly for your particular system by using the VirtualSetAttributes().  For example:


/* allocate dwBytes of virtual address space */


void *pVirtMem = VirtualAlloc(NULL,


                              dwBytes,


                              MEM_RESERVE,


                              0);


 


/* map the memory to physical address 0x10000000 (arbitrary) */


VirtualCopy(pVirtMem,


            (void *)(0x10000000>>8),


            dwBytes,


            PAGE_PHYSICAL | PAGE_READWRITE | PAGE_NOCACHE);


 


/* set the attributes on the pages */


VirtualSetAttributes(pVirtMem,


                     dwBytes,


                     0x13,


                     0x13,


                     NULL);


 


The attributes on the set of pages mapping the physical address would now have the bits 0x13 set, to allow for possible use of a specific bus that holds the physical peripheral. The use of 0x13 is arbitrary here, because these flags are CPU-architecture specific. Consult the appropriate documentation for your CPU architecture to learn what each bit does in your page table entry.  You might accidentally change the physical page number, which could cause some unexpected behavior! Caveats, and more information on the arguments to the virtual memory functions discussed here, can be found on MSDN help.


 


 

vapapic.bmp

Comments (30)

  1. milkyway says:

    Hello Kurt Kennett ,I have a question about the use of VirtualCopy.

    When should I use the PAGE_PHYSICAL with this function?When I use arbitrary physical address or physical address above 0x1FFFFFFF?

  2. Kurt Kennett says:

    PAGE_PHYSICAL’s use with VirtualCopy just specifies that the address that you’re giving at input is a Physical address, not another Virtual address.  You can copy a range of virtual-to-physical mappings where you do not care what the virtual memory maps to (in which case you would copy virtual->virtual).

  3. milkyway says:

    Kurt Kennett.thank for your answer.

    Does the use of VirtualCopy like this:

    We can map a physical space to virtual space with the parameter of PAGE_PHYSICAL,we can also map a virtual space of currently process  to the virtual space of kernel without PAGE_PHYSICAL?

    I’m confused by the VirtualCopy.Is the "source address" parameter can be both virtual space of kermel and physical address?

  4. Kurt Kennett says:

    VirtualCopy can be a bit confusing to use.

    http://msdn2.microsoft.com/en-us/library/aa908789.aspx

    Let’s look at the parameters.  I’ve commented with — after each of them.

    lpvDest

    [in] Pointer to the destination memory, which must be reserved.

    — This means that the destination range of VIRTUAL memory must already be reserved by a call to VirtualAlloc().  You must have allocated a range of virtual memory that you are going to map to some physical range.

    lpvSrc

    [in] Pointer to committed memory.

    — This is the range of either VIRTUAL or PHYSICAL memory that you want to map the range specified by the ‘lpvDest’ parameter to.  

    If you specify a VIRTUAL address and omit the PAGE_PHYSICAL flag from the fdwProtect parameter, then you are simply saying "Copy the mapping at the lpvSrc address to the lpvDest address".  This just means that there will be two ranges of virtual memory that point to the same physical range.

    If you specify a PHYSICAL address (shifted right 8 bits) and include the PAGE_PHYSICAL flag in the fdwProtect parameter, then you are saying "Map the range at the lpvDest address to this specific physical address".  This sets your new range of virtual memory to point to a piece of physical memory.

    cbSize

    [in] Size, in bytes, of the region. The allocated pages include all pages containing one or more bytes in the range from lpAddress to lpAddress+cbSize. This means that a 2-byte range straddling a page boundary causes both pages to be included in the allocated region.

    — pretty straight forward here.

    fdwProtect

    [in] Type of access protection. If the pages are being committed, any one of a number of flags can be specified, along with the PAGE_GUARD and PAGE_NOCACHE, protection modifier flags. The following table shows the flags that can be specified.

    — ‘Reserving’ a page means you’re allocating a range of virtual memory but not pointing it at anything yet.  ‘Commiting’ a page means you are actually taking up physical storage somewhere – be it in RAM or in physical addres space.  For the purposes of our discussion here, you would normally map registers and i/o space with PAGE_NOCACHE.  If you used a physical address (shifted right 8 bits) in the lpvSrc parameter, then you would also specify the PAGE_PHYSICAL flag.

    Hope this helps with your understanding.

  5. milkyway says:

    Thank you for your particular explaination!

    If I want to access some physical memory in my driver,  can I do like these way?

    Method (1) define static map relationship in OEMAddressTable and reserve difined virtual address in config.bib first, then use VirtualAlloc() and VirtualCopy() without the page_physical parameter.

    or (2)  directly use MmMapIoSpace() or VirtualAlloc+Copy() with the page_physical parameter.  

  6. Kurt Kennett says:

    Yes, either of those would work I believe.  #2 is the preferred/recommended method.

  7. milkyway says:

    Thank you for your help, kurt.

  8. Alex says:

    Hi,

    and what about other APIs like AllocPhysMem, BusTransBusAddrxxx,

    HalAllocateCommonBuffer, TransBusAddrToStatic, NKCreateStaticMapping/NKDeleteStaticMapping …

    So many APIs to do similar things (all with confusing MSDN API descriptions) deserve a good explanation. Do you take the challenge? 😉

    For example, not sure of the adventages of reserving a buffer area in the config.bib and which APIs I can use to access this area.

    Thanks a lot,

    Alex

  9. Kurt Kennett says:

    You raise a good point.  I will push for a separate blog article on the usage of the above APIs, as well as any other system-level memory APIs.

  10. linfan says:

    Hello Kurt,

            I have confusions about address mapping. When should I define static map relationship in OEMAddressTable and reserve difined virtual address in config.bib for device registers? If I use MmMapIospace( ) may I have to do these?

            As you explain the "lpvSrc" parameter of VirtualCopy( ), if I use it to map virual address , it means to map virual address defined in OEMAddressTable? The mapping’s function is to access some virtual address in user mode?

            And if I use VirtualCopy( ) to map physical address, it is almost do the same job as MmMapIospace( )? In this way, should I have to define static map relationship in OEMAddressTable ?

            Thanks for answer.

  11. Max says:

    Hello Kurt,

    I tried to use VirtualCopy with PAGE_EXECUTE_READWRITE but VirtualQuery returned PAGE_READWRITE, that is, the attribute of execution was discarded.

    Can you give an example to achieve the executable attribute?

    Thank for your answer.

  12. Kurt Kennett says:

    Wow!  Lots of interest in VirtualCopy!  Ok, I’ll try to be super-clear here.

    VirtualCopy copies or sets a range of virtual addresses.  

       – You use it to copy an existing Virtual->Physical mapping (no matter where it is).  

    OR

       – You use it to set a mapping to a range of physical addresses.

    In either case, the virtual memory you want to create the new map in must already be allocated (via VirtualAlloc()).

    OEMAddressTable is a static (unchanging, available at startup without doing any work or setup) table of virtual -> physical mappings.  The kernel is the only thing that has default access to the resources mapped by this table.  If you are operating outside the OAL (i.e. in any kind of driver or application), you must use VirtualCopy() to copy or create memory page mappings.  As mentioned above, you can copy any existing mapping as long as you have access to it.  This includes a static mapping done by the OEMAddressTable.  Some people will map all resources in the OEMAddressTable (so the kernel has access to everything), then just copy those mappings in drivers when they need to.  This is not a best practice because it makes driver code less portable — it is better to read the physical address of a component from the registry, then use the value found there to map to it. If you do this your driver code does not have to change if it is moved to a different platform or extended to use multiple components in different physical locations.

    A mapping does not have to exist in OEMAddressTable in order for you to access the physical resources mapped.  You can create a new mapping unknown to the OEMAddressTable by using VirtualCopy with the PAGE_PHYSICAL flag.

    MmMapIoSpace is a CEDDK function — it simply does the appropriate calls to VirtualAlloc/VirtualCopy.  You could write your own MmMapIoSpace if you wanted to.  Some people have in the past – calling the function "VirtualMemCopyPhysical" or something like that.  Some platforms need to modify the MmMapIoSpace to set Virtual mapping attributes when pages are mapped (hence the original purpose for this blog entry).

    Using flags like PAGE_EXECUTE_READWRITE with VirtualCopy are a ‘request’.  If the platform/CPU does not have a differentiation between executable pages and non-executable pages, the EXECUTE property will not be able to be set.  For example, the X86 CPUs can explicitly state that memory is executable, but can’t be read or written to.  The ARM processors have no notion of this – you can read it or read and write it.

    I hope this helps.

  13. linfan says:

    If I use VirtualAlloc() and VirtualCopy() without the page_physical parameter, I need to define static map relationship in OEMAddressTable first? But I don’t need to do anything in config.bib?

    And If I  use VirtualAlloc() and VirtualCopy() with the page_physical parameter, or I use MmMapIoSpace, I don’t need to define in OEMAddressTable?

    Thanks  you for making me clear.

  14. Kurt Kennett says:

    If you do not use PAGE_PHYSICAL, you are trying to copy an existing mapping (from somewhere).  So somebody else must have done a VirtualCopy with PAGE_PHYSICAL, or the mapping must already exist in the OEMAddressTable.  Config.bib reserves regions of memory that romimage knows about, but does not specify kernel memory regions.

    If you use PAGE_PHYSICAL, you don’t need an existing mapping in the OEMAddressTable.

  15. David says:

    Hi Kurt,

    The document said :

       For Windows Embedded CE 6.0, a call to VirtualCopy fails if it crosses a 32 MB section boundary. For example, if you allocate 16 MB and it happens to cross a 32 MB boundary, you will need to make two VirtualCopy calls.

    But how did I know whether my allocation crosses the 32MB section boundary?  For example, if I want to use MmMapIoSpace() to map an 50MB physical memory, should I divide it to 32MB and 22MB allocation (or 16MB, 16MB, 16MB and 6MB)?  Or I should use VirtualAlloc() and VirtualCopy() instead of MmMapIoSpace()?

    Thanks!

  16. Kurt Kennett says:

    Yes, in your case you should use VirtualAlloc(), VirtualCopy() to map the region, since you can’t know how MmMapIoSpace is going to do that alloc.  Either that or check/modify MmMapIoSpace to do the allocation so that it doesn’t cross a 32 meg boundary.

    Doing a VirtualAlloc() will give you a region of memory.  Find the 32 megabyte boundary inside that, then split it and do two VirtualCopy() calls, one for each part.

  17. ce_base says:

    I think this is a documentation error.  I think only CE5 and earlier had a limitation on 32MB boundaries.  In fact it might have only been up to CE4.2, I’m not sure.

    Sue

  18. jxy1101 says:

    Hi Sue,

    I think the document should be migrated from CE5  😉

    ===== VirtualCopy() begin =====

    For Windows CE 5.01, a call to VirtualCopy fails if it crosses a 32 MB section boundary. For example, if you allocate 16 MB and it happens to cross a 32 MB boundary, you will need to make two VirtualCopy calls.

    ===== VirtualCopy() end =====

    Hi Kurt,

    Did you mean the MmMapIoSpace() doesn’t proceed the 32MB boundary checking?  If it’s the case, every mappings/allocations via MmMapIoSpace() should have the chance to cross the 32MB boundary even I just want to map an 1MB physical address.

    For example, if someone calls MmMapIoSpace() to map an 1MB physical address, but inside MmMapIoSpace(), the VirtualAlloc() returns the virtual address crosses the 32MB boundary, then the VirtualCopy() will fail.

    Is my understanding correct?

    Thanks,

    David

  19. Kurt Kennett says:

    I will try to get clarity on this from the Kernel team.

  20. ce_base says:

    David,

    Sue is right. In CE 6.0 there is no such test on memory bondary checking for 32MB for VirtualCopy API. This was there in the previous releases mainly becasue of the slotized memory architecture.

    I also want to clarify few things mentioned before in this blog:

    — OEMAddressTable mentioned in this blog applies only to h/w based TLB designs like x86 and ARM. For SHx and MIPS, there is a architecture pre-defined mapping (512Mb cached and uncached regions) at bootup.

    — On ARM v6/v7 there is a bit (eXecute Never XN) which can be used to mark individual page entries. Once this is set, then any attempt to execute code from that page will fault. This most likely will be supported in future releases of CE.

    — There seemed to be lot of confusion (party our fault since there are so many ways you can map physical or virtual memory) on these APIs. In general remember that VirtualCopy can be used to create a virtual address mapped to either a physical address or another virtual address range. Also all the flags are well documented in MSDN so you should take a look at that.

    thx.

    -Upender

  21. jxy1101 says:

    Hi Upender,

    Thanks for your explain.  I understood CE6.0 has different memory architecture and it should be the documentation error.

    My questions are for CE5 (also for WM5 and WM6).

    Hi Kurt,

    Very very thanks for your great help! 🙂

    Thanks,

    David

  22. Fresh says:

    From DoVirtualCopy, I got following informations:

    When commit physical memory for a special virtual space sized 4096 bytes (one page), it will calculate which 4M space does this page lie in process’s space (suppose it’s second here). There also needs one physical page to save the page table for this 4M virtual space, the physical page’s address is saved in Process->pPTBL[1] ((4096/4)4096=4M). Then it will calculate which page does this page lie in this 4M space, we suppose it’s this second page in this 4M space, and the address of physical memory we have got for this virtual memory page is 0x56000000, it will write (0x56000000|(attribute bits)) as one page translation item to Process->pPTBL[ixTbl]+24.

    My question is that when I write a data in that page space, how does MMU works? Does the first page table and second page table MMU used have some relationship with Process->pPTBL[?].

  23. Kurt Kennett says:

    I’m sorry, but I don’t quite understand your question.  

    The internal workings of memory mapping inside the kernel are subject to change.  For a good understanding of how MMU mapping works, I suggest you acquire Andrew Sloss’ excellent book "ARM System Developer Guide" http://www.arm.com/documentation/books/4975.html

  24. milkyway says:

    Hi,

    I have a question: the virtual address space return by use of VirtualAlloc or MmMapIoSpace is inside the progress slot calling or beyond it (between 0x42000000-0x7E000000 insteadly).

  25. Kurt Kennett says:

    Hi Milkyway.  I assume you mean "process".

    A VirtualAlloc typically comes from the address space of the process which is calling it.

  26. ce_base says:

    I would like to correct Kurt’s answer.

    In Windows CE 5.0 and earlier, virtual allocations below 2MB come out of the address space of the process calling it, while allocations above 2MB come out of the shared address space.

    Sue

  27. milkyway says:

    Thanks,Kurt and Sue.

    And in Windows CE6.0, I can’t read/write physical address(such as buffers of devices) in my application, I should write driver in kernel virtual spaces to access them. Am I right?

  28. ce_base says:

    Clarification of my own statement:

    In Windows CE 5.0 and earlier, virtual allocations below 2MB in size will be allocated inside of the address space of the process calling it, while allocations above 2MB in size will be allocated out of the shared address space.  I was not talking about the address of the allocation, I was talking about the size.

    Sue

  29. ce_base says:

    And in CE 6.0, user-mode drivers can access physical addresses if they’re granted access in the registry.  Please read about our user mode driver model to decide if your needs can be satisfied by a user mode driver.  Many drivers that access physical devices are easier to write in kernel mode in CE 6.0, or may even require kernel mode.  User mode is more appropriate for very simple drivers or those that do not access physical devices.

    Sue

  30. jdenning says:

    I am curious as to what happens when you use MmUnmapIoSpace() with a bytes parameter of 0?  Is this lazy programming or does a specific and intentional action happen?  I do not see any mention of this in the MmUnmapIoSpace() documentation.

Skip to main content