identity mapped paging (Vaibhav Jain)

identity mapped paging (Vaibhav Jain)

[Pranay Kumar Srivastava]

> -----Original Message-----
> From: kernelnewbies-bounces at kernelnewbies.org [mailto:kernelnewbies-
> bounces at kernelnewbies.org] On Behalf Of kernelnewbies-
> request at kernelnewbies.org
> Sent: Thursday, April 12, 2012 3:47 PM
> To: kernelnewbies at kernelnewbies.org
> Subject: Kernelnewbies Digest, Vol 17, Issue 15
>
> Send Kernelnewbies mailing list submissions to
>       kernelnewbies at kernelnewbies.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
>       http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
> or, via email, send a message with subject or body 'help' to
>       kernelnewbies-request at kernelnewbies.org
>
> You can reach the person managing the list at
>       kernelnewbies-owner at kernelnewbies.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of Kernelnewbies digest..."
>
>
> Today's Topics:
>
>    1. Re: IRQs and memory consistency (Christopher Harvey)
>    2. identity mapped paging (Vaibhav Jain)
>    3. System Tap help (V.Ravikumar)
>    4. Re: System Tap help (V.Ravikumar)
>    5. Issue in uncompressing zImage (KARTHIK SEKURU)
>    6. [RFC]Something wrong with my module (harryxiyou)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Wed, 11 Apr 2012 15:14:44 -0500
> From: Christopher Harvey <chris@basementcode.com>
> Subject: Re: IRQs and memory consistency
> To: Wink Saville <wink@saville.com>
> Cc: kernelnewbies at kernelnewbies.org
> Message-ID: <a6554b231600d032628d59f975476e13@basementcode.com>
> Content-Type: text/plain; charset=UTF-8; format=flowed
>
> On 11.04.2012 10:35, Wink Saville wrote:
> > Do you have a read barrier in the IRQ?
>
> Ah, no I don't.
>
> > See "SMP BARRIER PAIRING" in:
> >
> > http://www.kernel.org/doc/Documentation/memory-barriers.txt [3]
>
> I saw that but only skimmed it.
>
> I'll let the mailing list know if it worked when I get a chance to try
> it.
>
> thanks,
> -Chris
>
>
>
>
> ------------------------------
>
> Message: 2
> Date: Wed, 11 Apr 2012 14:22:29 -0700
> From: Vaibhav Jain <vjoss197@gmail.com>
> Subject: identity mapped paging
> To: kernelnewbies at kernelnewbies.org
> Message-ID:
>       <CAKuUYSy-j5HogDpg0-
> 9j5RrVSPbNObBDKqafBEPgCTjHO2twFA at mail.gmail.com>
> Content-Type: text/plain; charset="iso-8859-1"
>
> I am not clear about the use of identity mapped paging while paging is
> being enabled by the operating system. Also I don't understand at what
> point are the
> identity mappings no longer useful.According to this article
> http://geezer.osdevbrasil.net/osd/mem/index.htm#identity - "The page
> table
> entries used to identity-map kernel memory can be deleted once paging
> and
> virtual addresses are enabled." Can somebody please explain?
>

Identity mapping is when VA(Virt Address)=PA(Physical address).

So basically when you set up your page tables you need to make sure they map identically. This is very easily done if you consider each 4KB block as a page beginning from location 0 upto whatever you've found to be the highest memory available either thru BIOS or GRUB.

Remember that while setting up your PTEs and PDE every address is a physical one. So if you thought that your kernel would be linked initially to a higher VA since you would remap it to a lower memory physically then that would be WRONG!. Without PTEs and PDEs installed don't do that!.

Why would you want it? Well for a simple reason, when your kernel starts to boot there's no translator,(No PTEs/PDEs and the Paging Enabled bit of processor is also cleared AFAIK just after the BIOS is done), yet since you've not enabled your processor for that but you'll be doing that in a moment.

So let's say you made your kernel to be linked to higher VA like 3Gigs. Now the addresses would be generated beginning 3Gigs however you still don't have the Page tables installed since your kernel just started. So in that case the address is the physical address. And if you've not loaded your kernel beginning 3Gigs then it would definitely come crashing down.

To avoid the crash in case you made your kernel to link to higher half of the memory, you can use GDT trick since segmentation is always on and you can make the overflow of the address addition to translate to a lower physical memory even if paging is not enabled yet. Thus it is possible to load the kernel at lower memory addresses while the linkage would be for higher VMA. And once your PTEs/PGD are enabled then you can use those instead of the GDT trick.

Here's a link to that http://wiki.osdev.org/Higher_Half_With_GDT



> Thanks
> Vaibhav Jain
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL:
> http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/2012
> 0411/ac8fccf1/attachment-0001.html
>
> ------------------------------
>
> Message: 3
> Date: Thu, 12 Apr 2012 10:38:15 +0530
> From: "V.Ravikumar" <ravikumar.vallabhu@gmail.com>
> Subject: System Tap help
> To: kernelnewbies at kernelnewbies.org,  Mulyadi Santosa
>       <mulyadi.santosa@gmail.com>
> Message-ID:
>       <CA+CftcG2xhjTq-
> REqM14Mv3dS5H4+FjrbyBGhZH1Xvz+Z8e=SQ at mail.gmail.com>
> Content-Type: text/plain; charset="iso-8859-1"
>
> Hi all,
> I'm probing sys_write system call using system tap to get modified file
> name along with UID.
> Here is code snippet.
>
> probe kernel.function ("vfs_write")
> {
>
>         filename = user_string($file->f_dentry->d_name->name)  // for
> 2.6.18 kernel.
>         printf ("%d %s\n",uid,filename)
> }
> I got sample program to get inode number as below
> inode_nr = $file->f_path->dentry->d_inode->i_ino . This I changed to
> get
> file name, I'm getting file name as empty.
>
> With above program i'm getting file name as unknown.
> My question is does above code snippet is correct to get file name ? If
> not
> could somebody please let me know the correct implementation.
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL:
> http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/2012
> 0412/784c4cd4/attachment-0001.html
>
> ------------------------------
>
> Message: 4
> Date: Thu, 12 Apr 2012 11:02:17 +0530
> From: "V.Ravikumar" <ravikumar.vallabhu@gmail.com>
> Subject: Re: System Tap help
> To: kernelnewbies at kernelnewbies.org,  Mulyadi Santosa
>       <mulyadi.santosa@gmail.com>
> Message-ID:
>       <CA+CftcH92ujFVM30+96eExJfrWLZKcGoxspkR=L=myh1a=n0+g@mail.gmail.c
> om>
> Content-Type: text/plain; charset="iso-8859-1"
>
> On Thu, Apr 12, 2012 at 10:38 AM, V.Ravikumar
> <ravikumar.vallabhu@gmail.com>wrote:
>
> > Hi all,
> > I'm probing sys_write system call using system tap to get modified
> file
> > name along with UID.
> > Here is code snippet.
> >
> > probe kernel.function ("vfs_write")
> > {
> >
> >         filename = user_string($file->f_dentry->d_name->name)  // for
> > 2.6.18 kernel.
> >         printf ("%d %s\n",uid,filename)
> > }
> > I got sample program to get inode number as below
> > inode_nr = $file->f_path->dentry->d_inode->i_ino . This I changed to
> get
> > file name, I'm getting file name as empty.
> >
> > With above program i'm getting file name as unknown.
> > My question is does above code snippet is correct to get file name ?
> If
> > not could somebody please let me know the correct implementation.
> >
> >
> I got how to get file name properly.
>
> > filename = user_string($file->f_dentry->d_name->name)  // for 2.6.18
>   Here I need to use kernel_string instead of user_string
>
> Regards,
> Ravi
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL:
> http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/2012
> 0412/4770e93f/attachment-0001.html
>
> ------------------------------
>
> Message: 5
> Date: Thu, 12 Apr 2012 15:37:37 +0530
> From: KARTHIK SEKURU <karthik.sekuru@gmail.com>
> Subject: Issue in uncompressing zImage
> To: kernelnewbies at kernelnewbies.org
> Message-ID:
>       <CANev7oU7fKhRX8eRO-
> F3=AgEtNS8Y6cqofksKooSg8ykV5Ku1w at mail.gmail.com>
> Content-Type: text/plain; charset="iso-8859-1"
>
> Hi,
> I'm doing kernel porting to arm926 based FPGA board,very minimal setup
> (uart + timer only)
> I'm getting kernel panic after cramfs root file system is mounted,I am
> expecting to enter login prompt after root file system is mounted.
>
> Below are the details
>
> #define PLAT_PHYS_OFFSET       0x80000000 (with 128 MB SDRAM)
> #define CONFIG_PAGE_OFFSET  0xc0000000
>
>
> Physical memory : 128MB (0x80000000 to 88000000)
>
> ZRELADDR = 0x80008000
> PARAMS_PHYS = 0x80000100
> INITRD_PHYS = 0x80800000
>
> Kernel cmd line : mem=32M console=ttySAC0,115200 root=/dev/ram0
> rootfstype=cramfs init=/linuxrc initrd=0x80800000,8M user_debug=31
>
>
>
>
>
> I am not using u-boot and doing all the required initializations
> through
> trace32 cmm script.
>
>
>
>
>
> After compiling the kernel, when I load vmlinux(through trace32) at
> 0x80008000, the kernel runs upto mounting the filesystem(though there
> is
> some issue in mounting the filesystem).
>
>
>
> But when I load zImage at the same address i.e 0x80008000, I am getting
>
>
>
> *Uncompressing Linux...*
>
> * *
>
> *uncompression error*
>
> * *
>
> * -- System halted*
>
>
>
> in the console.
>
>
>
> I made sure that I am enabling the config options CONFIG_XZ_DEC (xz
> decompression support) through menuconfig.
>
> I suspected on the cramfs memory location and then tried loading only
> zImage at 0x80008000.,even after this the result is same.
>
>
>
> Please let me know where could be the issue.
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL:
> http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/2012
> 0412/3a86415c/attachment-0001.html
>
> ------------------------------
>
> Message: 6
> Date: Thu, 12 Apr 2012 18:16:56 +0800
> From: harryxiyou <harryxiyou@gmail.com>
> Subject: [RFC]Something wrong with my module
> To: kernelnewbies at kernelnewbies.org
> Cc: Greg-Kroah-Hartman <gregkh@linuxfoundation.org>
> Message-ID:
>       <CAD+1EGO--aBWqehzmm+W1gX6wMD-
> d67P_Rpy97JTtbQ4U0VX=Q at mail.gmail.com>
> Content-Type: text/plain; charset=UTF-8
>
> Hi greg,
>
>     I write a module for inserting a PCB or delete a PCB to kernel's
> PCB tree, but when i run it something wrong happens to me like
> following.
> My environment is "Linux 10 2.6.35-22-generic #33-Ubuntu SMP Sun Sep
> 19 20:34:50 UTC 2010 i686 GNU/Linux"
>
> hw2.c
>
> #include <linux/module.h>
> #include <linux/kernel.h>
> #include <linux/init.h>
> #include <linux/sched.h>
> #include <linux/list.h>
> #include <linux/slab.h>
>
> struct pcb {
>       int pid;
>       int state;
>       int flag;
>       char *comm;
>       struct list_head tasks;
> };
>
> static int insert_task(struct task_struct *p) {
>       struct pcb *pcb1 = NULL;
>       pcb1 = (struct pcb *)kmalloc(sizeof(struct pcb), GFP_KERNEL);
>       if (NULL == pcb1) {
>               printk("<0> kmalloc failed!\n");
>       }
>       pcb1->state = 8;
>       pcb1->flag = 8;
>       pcb1->pid= 2;
>       pcb1->comm = "jiawei";
>       list_add(&pcb1->tasks, &p->tasks);
>       return 0;
> }
>
> static int rm_task(struct task_struct *p){
>       struct task_struct *del = p;
>       list_del(&p->tasks);
> //    kfree(del);
>       return 0;
> }
> #if 1
> static int print_pid(void) {
>       struct task_struct *task = NULL;
>       struct task_struct *p = NULL;
>       struct list_head *pos = NULL;
>       int count = 0;
>
>       printk("Search for insert task-------->\n");
>       task = &init_task;
>       list_for_each(pos, &task->tasks) {
>               p = list_entry(pos, struct task_struct, tasks);
>               count++;
>               if (0 == p->pid) {
>                       rm_task(p);
>               }
>               printk("pid: %d, state: %ld, comm: %s\n", p->pid, p->state,
> p->comm);
>       }
>       insert_task(p);
>       printk("the number of process is %d\n", count);
>       count = 0;
>       printk("show all tasks-------->\n");
>       task = &init_task;
>       list_for_each(pos, &task->tasks) {
>               p = list_entry(pos, struct task_struct, tasks);
>               count++;
>               printk("pid: %d, state: %ld, comm: %s\n", p->pid, p->state,
> p->comm);
>       }
>       printk("the number of process is %d\n", count);
>       return 0;
> }
> #endif
>
> static int __init hello_init(void) {
>       printk("<1> Hello World\n");
>       print_pid();
>       return 0;
> }
>
> static void __exit hello_exit(void) {
>       printk("<1> Goodbey\n");
>       return ;
> }
>
> MODULE_LICENSE("GPL");
> module_init(hello_init);
> module_exit(hello_exit);
>
>
> Dmesg logs:
>
> [ 1174.738305] Search for insert task-------->
> [ 1174.738308] pid: 1, state: 1, comm: init
> [ 1174.738312] pid: 2, state: 1, comm: kthreadd
> [ 1174.738316] pid: 3, state: 1, comm: ksoftirqd/0
> [ 1174.738319] pid: 4, state: 1, comm: migration/0
> [ 1174.738323] pid: 5, state: 1, comm: watchdog/0
> [ 1174.738326] pid: 6, state: 1, comm: events/0
> [ 1174.738329] pid: 7, state: 1, comm: cpuset
> [ 1174.738333] pid: 8, state: 1, comm: khelper
> [ 1174.738336] pid: 9, state: 1, comm: netns
> [ 1174.738340] pid: 10, state: 1, comm: async/mgr
> [ 1174.738343] pid: 11, state: 1, comm: pm
> [ 1174.738346] pid: 12, state: 1, comm: sync_supers
> [ 1174.738350] pid: 13, state: 1, comm: bdi-default
> [ 1174.738353] pid: 14, state: 1, comm: kintegrityd/0
> [ 1174.738357] pid: 15, state: 1, comm: kblockd/0
> [ 1174.738360] pid: 16, state: 1, comm: ata_aux
> [ 1174.738364] pid: 17, state: 1, comm: ata_sff/0
> [ 1174.738367] pid: 18, state: 1, comm: khubd
> [ 1174.738370] pid: 19, state: 1, comm: kseriod
> [ 1174.738374] pid: 20, state: 1, comm: kmmcd
> [ 1174.738377] pid: 23, state: 1, comm: kswapd0
> [ 1174.738380] pid: 24, state: 1, comm: ksmd
> [ 1174.738383] pid: 25, state: 1, comm: aio/0
> [ 1174.738387] pid: 26, state: 1, comm: ecryptfs-kthrea
> [ 1174.738390] pid: 27, state: 1, comm: crypto/0
> [ 1174.738394] pid: 33, state: 1, comm: scsi_eh_0
> [ 1174.738397] pid: 35, state: 1, comm: scsi_eh_1
> [ 1174.738400] pid: 38, state: 1, comm: kstriped
> [ 1174.738404] pid: 39, state: 1, comm: kmpathd/0
> [ 1174.738407] pid: 40, state: 1, comm: kmpath_handlerd
> [ 1174.738411] pid: 41, state: 1, comm: ksnapd
> [ 1174.738414] pid: 42, state: 1, comm: kondemand/0
> [ 1174.738417] pid: 43, state: 1, comm: kconservative/0
> [ 1174.738421] pid: 147, state: 1, comm: scsi_eh_2
> [ 1174.738424] pid: 148, state: 1, comm: scsi_eh_3
> [ 1174.738427] pid: 160, state: 1, comm: jbd2/sda1-8
> [ 1174.738431] pid: 161, state: 1, comm: ext4-dio-unwrit
> [ 1174.738434] pid: 194, state: 1, comm: flush-8:0
> [ 1174.738438] pid: 223, state: 1, comm: upstart-udev-br
> [ 1174.738441] pid: 225, state: 1, comm: udevd
> [ 1174.738445] pid: 284, state: 1, comm: udevd
> [ 1174.738448] pid: 289, state: 1, comm: udevd
> [ 1174.738451] pid: 323, state: 1, comm: kpsmoused
> [ 1174.738455] pid: 477, state: 1, comm: hd-audio0
> [ 1174.738458] pid: 502, state: 1, comm: sshd
> [ 1174.738462] pid: 506, state: 1, comm: rsyslogd
> [ 1174.738465] pid: 529, state: 1, comm: dbus-daemon
> [ 1174.738469] pid: 561, state: 1, comm: gdm-binary
> [ 1174.738472] pid: 566, state: 1, comm: NetworkManager
> [ 1174.738476] pid: 577, state: 1, comm: avahi-daemon
> [ 1174.738479] pid: 586, state: 1, comm: avahi-daemon
> [ 1174.738483] pid: 587, state: 1, comm: modem-manager
> [ 1174.738486] pid: 597, state: 1, comm: console-kit-dae
> [ 1174.738490] pid: 675, state: 1, comm: gdm-simple-slav
> [ 1174.738493] pid: 679, state: 1, comm: cupsd
> [ 1174.738497] pid: 684, state: 1, comm: wpa_supplicant
> [ 1174.738501] pid: 688, state: 1, comm: dhclient
> [ 1174.738504] pid: 707, state: 1, comm: Xorg
> [ 1174.738507] pid: 742, state: 1, comm: getty
> [ 1174.738511] pid: 748, state: 1, comm: getty
> [ 1174.738514] pid: 758, state: 1, comm: getty
> [ 1174.738517] pid: 760, state: 1, comm: getty
> [ 1174.738520] pid: 764, state: 1, comm: getty
> [ 1174.738524] pid: 766, state: 1, comm: acpid
> [ 1174.738527] pid: 771, state: 1, comm: cron
> [ 1174.738530] pid: 772, state: 1, comm: atd
> [ 1174.738534] pid: 909, state: 1, comm: dbus-launch
> [ 1174.738537] pid: 978, state: 1, comm: gdm-session-wor
> [ 1174.738541] pid: 986, state: 1, comm: upowerd
> [ 1174.738544] pid: 990, state: 1, comm: rtkit-daemon
> [ 1174.738548] pid: 1000, state: 1, comm: polkitd
> [ 1174.738551] pid: 1071, state: 1, comm: getty
> [ 1174.738555] pid: 1082, state: 1, comm: gnome-keyring-d
> [ 1174.738559] pid: 1101, state: 1, comm: gnome-session
> [ 1174.738562] pid: 1128, state: 1, comm: ibus-daemon
> [ 1174.738566] pid: 1132, state: 1, comm: ssh-agent
> [ 1174.738569] pid: 1135, state: 1, comm: dbus-launch
> [ 1174.738573] pid: 1136, state: 1, comm: dbus-daemon
> [ 1174.738576] pid: 1142, state: 1, comm: gconfd-2
> [ 1174.738580] pid: 1146, state: 1, comm: gnome-power-man
> [ 1174.738583] pid: 1152, state: 1, comm: gnome-settings-
> [ 1174.738587] pid: 1157, state: 1, comm: gvfsd
> [ 1174.738590] pid: 1162, state: 1, comm: gvfs-fuse-daemo
> [ 1174.738593] pid: 1170, state: 1, comm: polkit-gnome-au
> [ 1174.738597] pid: 1172, state: 1, comm: pulseaudio
> [ 1174.738600] pid: 1173, state: 1, comm: bluetooth-apple
> [ 1174.738604] pid: 1176, state: 1, comm: evolution-alarm
> [ 1174.738608] pid: 1177, state: 1, comm: nm-applet
> [ 1174.738611] pid: 1178, state: 1, comm: gnome-panel
> [ 1174.738615] pid: 1180, state: 1, comm: metacity
> [ 1174.738618] pid: 1182, state: 1, comm: nautilus
> [ 1174.738621] pid: 1189, state: 1, comm: gconf-helper
> [ 1174.738625] pid: 1202, state: 1, comm: ibus-gconf
> [ 1174.738629] pid: 1204, state: 1, comm: python
> [ 1174.738632] pid: 1206, state: 1, comm: ibus-x11
> [ 1174.738635] pid: 1214, state: 1, comm: ibus-engine-pin
> [ 1174.738639] pid: 1226, state: 1, comm: gvfs-gdu-volume
> [ 1174.738642] pid: 1249, state: 1, comm: udisks-daemon
> [ 1174.738646] pid: 1271, state: 1, comm: udisks-daemon
> [ 1174.738649] pid: 1279, state: 1, comm: gvfs-afc-volume
> [ 1174.738653] pid: 1281, state: 1, comm: gvfsd-trash
> [ 1174.738656] pid: 1284, state: 1, comm: gvfs-gphoto2-vo
> [ 1174.738660] pid: 1289, state: 1, comm: gnome-screensav
> [ 1174.738663] pid: 1291, state: 1, comm: bonobo-activati
> [ 1174.738667] pid: 1299, state: 1, comm: wnck-applet
> [ 1174.738670] pid: 1301, state: 1, comm: trashapplet
> [ 1174.738674] pid: 1313, state: 1, comm: indicator-apple
> [ 1174.738677] pid: 1317, state: 1, comm: clock-applet
> [ 1174.738681] pid: 1318, state: 1, comm: indicator-apple
> [ 1174.738684] pid: 1319, state: 1, comm: notification-ar
> [ 1174.738688] pid: 1329, state: 1, comm: indicator-me-se
> [ 1174.738691] pid: 1333, state: 1, comm: indicator-sessi
> [ 1174.738695] pid: 1337, state: 1, comm: gvfsd-burn
> [ 1174.738698] pid: 1343, state: 1, comm: indicator-messa
> [ 1174.738702] pid: 1347, state: 1, comm: indicator-appli
> [ 1174.738705] pid: 1349, state: 1, comm: indicator-sound
> [ 1174.738709] pid: 1350, state: 1, comm: gdu-notificatio
> [ 1174.738712] pid: 1353, state: 1, comm: gnome-terminal
> [ 1174.738716] pid: 1356, state: 1, comm: gnome-pty-helpe
> [ 1174.738719] pid: 1357, state: 1, comm: bash
> [ 1174.738722] pid: 1381, state: 1, comm: firefox
> [ 1174.738726] pid: 1385, state: 1, comm: run-mozilla.sh
> [ 1174.738729] pid: 1389, state: 1, comm: firefox-bin
> [ 1174.738733] pid: 1419, state: 1, comm: applet.py
> [ 1174.738736] pid: 1427, state: 1, comm: sshd
> [ 1174.738740] pid: 1519, state: 1, comm: sshd
> [ 1174.738743] pid: 1520, state: 1, comm: bash
> [ 1174.738746] pid: 1557, state: 1, comm: update-notifier
> [ 1174.738750] pid: 1561, state: 1, comm: sshd
> [ 1174.738753] pid: 1657, state: 1, comm: sshd
> [ 1174.738756] pid: 1658, state: 1, comm: bash
> [ 1174.738760] pid: 1671, state: 1, comm: sshd
> [ 1174.738763] pid: 1784, state: 1, comm: sshd
> [ 1174.738766] pid: 1785, state: 1, comm: bash
> [ 1174.738769] pid: 1812, state: 1, comm: sshd
> [ 1174.738773] pid: 1911, state: 1, comm: sshd
> [ 1174.738776] pid: 1912, state: 1, comm: bash
> [ 1174.738779] pid: 1925, state: 1, comm: sshd
> [ 1174.738783] pid: 2038, state: 1, comm: sshd
> [ 1174.738786] pid: 2039, state: 1, comm: bash
> [ 1174.738789] pid: 2170, state: 1, comm: su
> [ 1174.738792] pid: 2177, state: 1, comm: bash
> [ 1174.738796] pid: 2285, state: 1, comm: vim
> [ 1174.738799] pid: 2319, state: 1, comm: vim
> [ 1174.738802] pid: 2320, state: 1, comm: sshd
> [ 1174.738806] pid: 2413, state: 1, comm: bash
> [ 1174.738809] pid: 2456, state: 1, comm: man
> [ 1174.738812] pid: 2465, state: 1, comm: pager
> [ 1174.738816] pid: 2474, state: 1, comm: su
> [ 1174.738819] pid: 2481, state: 1, comm: bash
> [ 1174.738822] pid: 0, state: 1, comm:
> [ 1174.738840] BUG: unable to handle kernel paging request at 00100100
> [ 1174.738845] IP: [<f072a08d>] hello_init+0x8d/0x1c8 [hw2]
> [ 1174.738853] *pde = 00000000
> [ 1174.738857] Oops: 0000 [#3] SMP
> [ 1174.738860] last sysfs file:
> /sys/devices/pci0000:00/0000:00:03.3/usb1/1-0:1.0/uevent
> [ 1174.738865] Modules linked in: hw2(+) hw1(+) binfmt_misc parport_pc
> ppdev snd_hda_codec_realtek snd_hda_intel snd_hda_codec snd_hwdep
> snd_pcm snd_seq_midi snd_rawmidi snd_seq_midi_event snd_seq snd_timer
> snd_seq_device snd sis_agp psmouse serio_raw soundcore snd_page_alloc
> shpchp agpgart lp parport 8139too 8139cp sata_sis mii [last unloaded:
> hw1]
> [ 1174.738895]
> [ 1174.738900] Pid: 4340, comm: insmod Tainted: G      D
> 2.6.35-22-generic #33-Ubuntu /
> [ 1174.738904] EIP: 0060:[<f072a08d>] EFLAGS: 00010292 CPU: 0
> [ 1174.738908] EIP is at hello_init+0x8d/0x1c8 [hw2]
> [ 1174.738911] EAX: 0000002b EBX: 00100100 ECX: c07cdd3c EDX: 00000000
> [ 1174.738914] ESI: d85650e0 EDI: 00000097 EBP: d8781f5c ESP: d8781f34
> [ 1174.738917]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
> [ 1174.738921] Process insmod (pid: 4340, ti=d8780000 task=ef098000
> task.ti=d8780000)
> [ 1174.738923] Stack:
> [ 1174.738925]  f07210d8 00000000 00000001 d85653d0 00000001 c07d0f50
> 00000001 096bb018
> [ 1174.738933] <0> f0721120 00000000 d8781f88 c0101132 f0721120
> c07d0f60 096bb018 f0721120
> [ 1174.738940] <0> 00004000 f072a000 096bb018 f0721120 00004000
> d8781fac c0180c2b 00000003
> [ 1174.738948] Call Trace:
> [ 1174.738958]  [<c0101132>] ? do_one_initcall+0x32/0x1a0
> [ 1174.738963]  [<f072a000>] ? hello_init+0x0/0x1c8 [hw2]
> [ 1174.738969]  [<c0180c2b>] ? sys_init_module+0x9b/0x1e0
> [ 1174.738975]  [<c02169c5>] ? sys_close+0x75/0xc0
> [ 1174.738982]  [<c05c90a4>] ? syscall_call+0x7/0xb
> [ 1174.738984] Code: 00 00 00 02 20 00 8b 16 8d 8e f0 02 00 00 83 c7
> 01 89 4c 24 0c 89 44 24 04 c7 04 24 d8 10 72 f0 89 54 24 08 e8 b0 c3
> e9 cf 8b 1b <8b> 03 0f 18 00 90 81 fb 90 78 7c c0 75 92 ba d0 00 00 00
> b8 68
> [ 1174.739024] EIP: [<f072a08d>] hello_init+0x8d/0x1c8 [hw2] SS:ESP
> 0068:d8781f34
> [ 1174.739030] CR2: 0000000000100100
> [ 1174.739035] ---[ end trace babb4642fde28352 ]---
> [ 1620.965086] ------------[ cut here ]------------
> [ 1620.965105] WARNING: at
> /build/buildd/linux-2.6.35/net/sched/sch_generic.c:258
> dev_watchdog+0x1fd/0x210()
> [ 1620.965109] NETDEV WATCHDOG: eth1 (8139too): transmit queue 0 timed
> out
> [ 1620.965112] Modules linked in: hw2(+) hw1(+) binfmt_misc parport_pc
> ppdev snd_hda_codec_realtek snd_hda_intel snd_hda_codec snd_hwdep
> snd_pcm snd_seq_midi snd_rawmidi snd_seq_midi_event snd_seq snd_timer
> snd_seq_device snd sis_agp psmouse serio_raw soundcore snd_page_alloc
> shpchp agpgart lp parport 8139too 8139cp sata_sis mii [last unloaded:
> hw1]
> [ 1620.965147] Pid: 0, comm: swapper Tainted: G      D
> 2.6.35-22-generic #33-Ubuntu
> [ 1620.965150] Call Trace:
> [ 1620.965160]  [<c014ac52>] warn_slowpath_common+0x72/0xa0
> [ 1620.965166]  [<c050e62d>] ? dev_watchdog+0x1fd/0x210
> [ 1620.965170]  [<c050e62d>] ? dev_watchdog+0x1fd/0x210
> [ 1620.965175]  [<c014ad23>] warn_slowpath_fmt+0x33/0x40
> [ 1620.965179]  [<c050e62d>] dev_watchdog+0x1fd/0x210
> [ 1620.965185]  [<c0108a28>] ? sched_clock+0x8/0x10
> [ 1620.965191]  [<c016bf14>] ? sched_clock_local+0xa4/0x180
> [ 1620.965195]  [<c050e430>] ? dev_watchdog+0x0/0x210
> [ 1620.965200]  [<c0157e1f>] call_timer_fn+0x2f/0xf0
> [ 1620.965204]  [<c0169a5a>] ? hrtimer_run_pending+0x4a/0xe0
> [ 1620.965209]  [<c0159064>] run_timer_softirq+0x104/0x210
> [ 1620.965213]  [<c016c121>] ? sched_clock_cpu+0x131/0x190
> [ 1620.965217]  [<c050e430>] ? dev_watchdog+0x0/0x210
> [ 1620.965222]  [<c015127c>] __do_softirq+0x9c/0x1b0
> [ 1620.965226]  [<c0158af3>] ? update_process_times+0x53/0x70
> [ 1620.965230]  [<c01513d5>] do_softirq+0x45/0x50
> [ 1620.965234]  [<c0151545>] irq_exit+0x65/0x70
> [ 1620.965239]  [<c05cf72b>] smp_apic_timer_interrupt+0x5b/0x8a
> [ 1620.965243]  [<c016c1ce>] ? sched_clock_idle_sleep_event+0xe/0x10
> [ 1620.965248]  [<c05c9535>] apic_timer_interrupt+0x31/0x38
> [ 1620.965254]  [<c010a766>] ? mwait_idle+0x56/0xb0
> [ 1620.965257]  [<c0101fcc>] cpu_idle+0x8c/0xd0
> [ 1620.965262]  [<c05b2431>] rest_init+0x71/0x80
> [ 1620.965269]  [<c081981a>] start_kernel+0x36e/0x374
> [ 1620.965273]  [<c08199dd>] ? pass_all_bootoptions+0x0/0xa
> [ 1620.965277]  [<c08190d7>] i386_start_kernel+0xd7/0xdf
> [ 1620.965280] ---[ end trace babb4642fde28353 ]---
>
> Cloud you please give me some help?
>
> --
> Thanks
> Harry Wei
>
>
>
> ------------------------------
>
> _______________________________________________
> Kernelnewbies mailing list
> Kernelnewbies at kernelnewbies.org
> http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
>
>
> End of Kernelnewbies Digest, Vol 17, Issue 15
> *********************************************

::DISCLAIMER::
-----------------------------------------------------------------------------------------------------------------------

The contents of this e-mail and any attachment(s) are confidential and intended for the named recipient(s) only.
It shall not attach any liability on the originator or HCL or its affiliates. Any views or opinions presented in
this email are solely those of the author and may not necessarily reflect the opinions of HCL or its affiliates.
Any form of reproduction, dissemination, copying, disclosure, modification, distribution and / or publication of
this message without the prior written consent of the author of this e-mail is strictly prohibited. If you have
received this email in error please delete it and notify the sender immediately. Before opening any mail and
attachments please check them for viruses and defect.

-----------------------------------------------------------------------------------------------------------------------

identity mapped paging (Vaibhav Jain)

[Vaibhav Jain]

>
> > I am not clear about the use of identity mapped paging while paging is
> > being enabled by the operating system. Also I don't understand at what
> > point are the
> > identity mappings no longer useful.According to this article
> > http://geezer.osdevbrasil.net/osd/mem/index.htm#identity - "The page
> > table
> > entries used to identity-map kernel memory can be deleted once paging
> > and
> > virtual addresses are enabled." Can somebody please explain?
> >
>
> Identity mapping is when VA(Virt Address)=PA(Physical address).
>
> So basically when you set up your page tables you need to make sure they
> map identically. This is very easily done if you consider each 4KB block as
> a page beginning from location 0 upto whatever you've found to be the
> highest memory available either thru BIOS or GRUB.
>
> Remember that while setting up your PTEs and PDE every address is a
> physical one. So if you thought that your kernel would be linked initially
> to a higher VA since you would remap it to a lower memory physically then
> that would be WRONG!. Without PTEs and PDEs installed don't do that!.
>
> Why would you want it? Well for a simple reason, when your kernel starts
> to boot there's no translator,(No PTEs/PDEs and the Paging Enabled bit of
> processor is also cleared AFAIK just after the BIOS is done), yet since
> you've not enabled your processor for that but you'll be doing that in a
> moment.
>
> So let's say you made your kernel to be linked to higher VA like 3Gigs.
> Now the addresses would be generated beginning 3Gigs however you still
> don't have the Page tables installed since your kernel just started. So in
> that case the address is the physical address. And if you've not loaded
> your kernel beginning 3Gigs then it would definitely come crashing down.
>
> To avoid the crash in case you made your kernel to link to higher half of
> the memory, you can use GDT trick since segmentation is always on and you
> can make the overflow of the address addition to translate to a lower
> physical memory even if paging is not enabled yet. Thus it is possible to
> load the kernel at lower memory addresses while the linkage would be for
> higher VMA. And once your PTEs/PGD are enabled then you can use those
> instead of the GDT trick.
>
> Here's a link to that http://wiki.osdev.org/Higher_Half_With_GDT
>
> > Thanks
> > Vaibhav Jain
>


Hi,

Thanks for replying but I am still confused. I continued reading about this
thing and what
I have understood is the following :
After the kernel executes the instruction to enable paging the instruction
pointer will contain the
address of the next instruction which will now be treated as a virtual
address. So for the next instruction to be executed
the page table should map this address to itself.
Please correct me if I am wrong.
I am confused by the point about linking  the kernel to higher address.
Could you please put that in a step by step manner
to make it clear what  happens before paging is enabled and what happens
after that.
Also, please explain at what point during the execution of kernel code are
the identity-mapped addresses no longer useful ?




Thanks
Vaibhav
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20120413/62344e48/attachment.html 

identity mapped paging (Vaibhav Jain)

[Vladimir Murzin]

On Fri, Apr 13, 2012 at 02:15:20AM -0700, Vaibhav Jain wrote:
> >
> > > I am not clear about the use of identity mapped paging while paging is
> > > being enabled by the operating system. Also I don't understand at what
> > > point are the
> > > identity mappings no longer useful.According to this article
> > > http://geezer.osdevbrasil.net/osd/mem/index.htm#identity - "The page
> > > table
> > > entries used to identity-map kernel memory can be deleted once paging
> > > and
> > > virtual addresses are enabled." Can somebody please explain?
> > >
> >
> > Identity mapping is when VA(Virt Address)=PA(Physical address).
> >
> > So basically when you set up your page tables you need to make sure they
> > map identically. This is very easily done if you consider each 4KB block as
> > a page beginning from location 0 upto whatever you've found to be the
> > highest memory available either thru BIOS or GRUB.
> >
> > Remember that while setting up your PTEs and PDE every address is a
> > physical one. So if you thought that your kernel would be linked initially
> > to a higher VA since you would remap it to a lower memory physically then
> > that would be WRONG!. Without PTEs and PDEs installed don't do that!.
> >
> > Why would you want it? Well for a simple reason, when your kernel starts
> > to boot there's no translator,(No PTEs/PDEs and the Paging Enabled bit of
> > processor is also cleared AFAIK just after the BIOS is done), yet since
> > you've not enabled your processor for that but you'll be doing that in a
> > moment.
> >
> > So let's say you made your kernel to be linked to higher VA like 3Gigs.
> > Now the addresses would be generated beginning 3Gigs however you still
> > don't have the Page tables installed since your kernel just started. So in
> > that case the address is the physical address. And if you've not loaded
> > your kernel beginning 3Gigs then it would definitely come crashing down.
> >
> > To avoid the crash in case you made your kernel to link to higher half of
> > the memory, you can use GDT trick since segmentation is always on and you
> > can make the overflow of the address addition to translate to a lower
> > physical memory even if paging is not enabled yet. Thus it is possible to
> > load the kernel at lower memory addresses while the linkage would be for
> > higher VMA. And once your PTEs/PGD are enabled then you can use those
> > instead of the GDT trick.
> >
> > Here's a link to that http://wiki.osdev.org/Higher_Half_With_GDT
> >
> > > Thanks
> > > Vaibhav Jain
> >
> 
> 
> Hi,
> 
> Thanks for replying but I am still confused. I continued reading about this
> thing and what
> I have understood is the following :
> After the kernel executes the instruction to enable paging the instruction
> pointer will contain the
> address of the next instruction which will now be treated as a virtual
> address. So for the next instruction to be executed
> the page table should map this address to itself.
> Please correct me if I am wrong.
> I am confused by the point about linking  the kernel to higher address.
> Could you please put that in a step by step manner
> to make it clear what  happens before paging is enabled and what happens
> after that.
> Also, please explain at what point during the execution of kernel code are
> the identity-mapped addresses no longer useful ?
> 
> 
> 
> 
> Thanks
> Vaibhav

> _______________________________________________
> Kernelnewbies mailing list
> Kernelnewbies at kernelnewbies.org
> http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Hi

May be notes in [1] make it a bit clear ;)

[1] http://pankaj-techstuff.blogspot.com/2007/12/initialization-of-arm-mmu-in-linux.html

Best wishes
Vladimir Murzin

identity mapped paging (Vaibhav Jain)

[Vaibhav Jain]

On Fri, Apr 13, 2012 at 2:15 AM, Vaibhav Jain <vjoss197@gmail.com> wrote:

>
>
>> > I am not clear about the use of identity mapped paging while paging is
>> > being enabled by the operating system. Also I don't understand at what
>> > point are the
>> > identity mappings no longer useful.According to this article
>> > http://geezer.osdevbrasil.net/osd/mem/index.htm#identity - "The page
>> > table
>> > entries used to identity-map kernel memory can be deleted once paging
>> > and
>> > virtual addresses are enabled." Can somebody please explain?
>> >
>>
>> Identity mapping is when VA(Virt Address)=PA(Physical address).
>>
>> So basically when you set up your page tables you need to make sure they
>> map identically. This is very easily done if you consider each 4KB block as
>> a page beginning from location 0 upto whatever you've found to be the
>> highest memory available either thru BIOS or GRUB.
>>
>> Remember that while setting up your PTEs and PDE every address is a
>> physical one. So if you thought that your kernel would be linked initially
>> to a higher VA since you would remap it to a lower memory physically then
>> that would be WRONG!. Without PTEs and PDEs installed don't do that!.
>>
>> Why would you want it? Well for a simple reason, when your kernel starts
>> to boot there's no translator,(No PTEs/PDEs and the Paging Enabled bit of
>> processor is also cleared AFAIK just after the BIOS is done), yet since
>> you've not enabled your processor for that but you'll be doing that in a
>> moment.
>>
>> So let's say you made your kernel to be linked to higher VA like 3Gigs.
>> Now the addresses would be generated beginning 3Gigs however you still
>> don't have the Page tables installed since your kernel just started. So in
>> that case the address is the physical address. And if you've not loaded
>> your kernel beginning 3Gigs then it would definitely come crashing down.
>>
>> To avoid the crash in case you made your kernel to link to higher half of
>> the memory, you can use GDT trick since segmentation is always on and you
>> can make the overflow of the address addition to translate to a lower
>> physical memory even if paging is not enabled yet. Thus it is possible to
>> load the kernel at lower memory addresses while the linkage would be for
>> higher VMA. And once your PTEs/PGD are enabled then you can use those
>> instead of the GDT trick.
>>
>> Here's a link to that http://wiki.osdev.org/Higher_Half_With_GDT
>>
>> > Thanks
>> > Vaibhav Jain
>>
>
>
> Hi,
>
> Thanks for replying but I am still confused. I continued reading about
> this thing and what
> I have understood is the following :
> After the kernel executes the instruction to enable paging the instruction
> pointer will contain the
> address of the next instruction which will now be treated as a virtual
> address. So for the next instruction to be executed
> the page table should map this address to itself.
> Please correct me if I am wrong.
> I am confused by the point about linking  the kernel to higher address.
> Could you please put that in a step by step manner
> to make it clear what  happens before paging is enabled and what happens
> after that.
> Also, please explain at what point during the execution of kernel code are
> the identity-mapped addresses no longer useful ?
>
>
>
>
> Thanks
> Vaibhav
>
Hi,

I am somewhat understanding your point. But I have some other queries now
in my mind.

If the kernel is linked to 3Gigs is there a way other than the GDT trick.?

In fact I am wondering that if the kernel is linked to 3Gigs and Grub loads
it at 1MB physical, how will even the first instruction of kernel
execute ?  I mean if all the address generated by kernel are above 3 Gigs
and paging is not enabled how will it start
running ?

Thanks
Vaibhav Jain
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20120417/dfdaa0be/attachment-0001.html 

identity mapped paging (Vaibhav Jain)

[Pranay Kumar Srivastava]

> -----Original Message-----
> From: Vaibhav Jain [mailto:vjoss197 at gmail.com]
> Sent: Tuesday, April 17, 2012 4:07 PM
> To: Pranay Kumar Srivastava
> Cc: kernelnewbies at kernelnewbies.org
> Subject: Re: identity mapped paging (Vaibhav Jain)
>
>
> On Fri, Apr 13, 2012 at 2:15 AM, Vaibhav Jain <vjoss197@gmail.com>
> wrote:
>
>
> > I am not clear about the use of identity mapped paging while paging
> is
> > being enabled by the operating system. Also I don't understand at
> what
> > point are the
> > identity mappings no longer useful.According to this article
> > http://geezer.osdevbrasil.net/osd/mem/index.htm#identity - "The page
> > table
> > entries used to identity-map kernel memory can be deleted once paging
> > and
> > virtual addresses are enabled." Can somebody please explain?
> >
>
> Identity mapping is when VA(Virt Address)=PA(Physical address).
>
> So basically when you set up your page tables you need to make sure
> they map identically. This is very easily done if you consider each 4KB
> block as a page beginning from location 0 upto whatever you've found to
> be the highest memory available either thru BIOS or GRUB.
>
> Remember that while setting up your PTEs and PDE every address is a
> physical one. So if you thought that your kernel would be linked
> initially to a higher VA since you would remap it to a lower memory
> physically then that would be WRONG!. Without PTEs and PDEs installed
> don't do that!.
>
> Why would you want it? Well for a simple reason, when your kernel
> starts to boot there's no translator,(No PTEs/PDEs and the Paging
> Enabled bit of processor is also cleared AFAIK just after the BIOS is
> done), yet since you've not enabled your processor for that but you'll
> be doing that in a moment.
>
> So let's say you made your kernel to be linked to higher VA like 3Gigs.
> Now the addresses would be generated beginning 3Gigs however you still
> don't have the Page tables installed since your kernel just started. So
> in that case the address is the physical address. And if you've not
> loaded your kernel beginning 3Gigs then it would definitely come
> crashing down.
>
> To avoid the crash in case you made your kernel to link to higher half
> of the memory, you can use GDT trick since segmentation is always on
> and you can make the overflow of the address addition to translate to a
> lower physical memory even if paging is not enabled yet. Thus it is
> possible to load the kernel at lower memory addresses while the linkage
> would be for higher VMA. And once your PTEs/PGD are enabled then you
> can use those instead of the GDT trick.
>
> Here's a link to that http://wiki.osdev.org/Higher_Half_With_GDT
>
> > Thanks
> > Vaibhav Jain
>
> Hi,
>
> Thanks for replying but I am still confused. I continued reading about
> this thing and what
> I have understood is the following :
> After the kernel executes the instruction to enable paging the
> instruction pointer will contain the
> address of the next instruction which will now be treated as a virtual
> address. So for the next instruction to be executed
> the page table should map this address to itself.
> Please correct me if I am wrong.
> I am confused by the point about linking  the kernel to higher address.
> Could you please put that in a step by step manner
> to make it clear what  happens before paging is enabled and what
> happens after that.
> Also, please explain at what point during the execution of kernel code
> are the identity-mapped addresses no longer useful ?
>
>
>
>
> Thanks
> Vaibhav
> Hi,
>
> I am somewhat understanding your point. But I have some other queries
> now in my mind.
>
> If the kernel is linked to 3Gigs is there a way other than the GDT
> trick.?

Make your load address = VA when you link so you won't have to worry about doing the GDT trick.

>
> In fact I am wondering that if the kernel is linked to 3Gigs and Grub
> loads it at 1MB physical, how will even the first instruction of kernel
> execute ?  I mean if all the address generated by kernel are above 3
> Gigs and paging is not enabled how will it start
> running ?

That's what the GDT trick is for. If you read the intel/amd processor manuals the segmentation is always on. So when the address get generated your segment's base address is still added to the generated address before it is put on wire. You can add a constant offset (in your GDT's base address part) to the generated address to get the address beginning from the load address of your kernel.

I would suggest you make the higher half kernel later and try to first create some code that can fragment your available memory into pages and store this information so you'll know what all pages are there. Next would be to do identity mapping, since your kernel VMA=LMA in your linker script this would be easier to do.

When you get that paging enabled you can move on to higher half kernel. I would suggest you to work on page replacement algos and virtual memory management code side by side for better integration with paging in later stages.

Maybe you can post your code if you are allowed to then I can have a look at it.

>
> Thanks
> Vaibhav Jain

::DISCLAIMER::
-----------------------------------------------------------------------------------------------------------------------

The contents of this e-mail and any attachment(s) are confidential and intended for the named recipient(s) only.
It shall not attach any liability on the originator or HCL or its affiliates. Any views or opinions presented in
this email are solely those of the author and may not necessarily reflect the opinions of HCL or its affiliates.
Any form of reproduction, dissemination, copying, disclosure, modification, distribution and / or publication of
this message without the prior written consent of the author of this e-mail is strictly prohibited. If you have
received this email in error please delete it and notify the sender immediately. Before opening any mail and
attachments please check them for viruses and defect.

-----------------------------------------------------------------------------------------------------------------------

identity mapped paging (Vaibhav Jain)

[Liu Zhiyou]

Hi,

I am not very sure what you guys are talking about. I guess we are
discussing the moment enabling paging.

On 2012-4-15 10:49, Vladimir Murzin wrote:
>  On Fri, Apr 13, 2012 at 02:15:20AM -0700, Vaibhav Jain wrote:
>>>>  I am not clear about the use of identity mapped paging while paging is
>>>>  being enabled by the operating system. Also I don't understand at what
>>>>  point are the
>>>>  identity mappings no longer useful.According to this article
>>>>  http://geezer.osdevbrasil.net/osd/mem/index.htm#identity  - "The page
>>>>  table
>>>>  entries used to identity-map kernel memory can be deleted once paging
>>>>  and
>>>>  virtual addresses are enabled." Can somebody please explain?
>>>>
>>>  Identity mapping is when VA(Virt Address)=PA(Physical address).
>>>
>>>  So basically when you set up your page tables you need to make sure they
>>>  map identically. This is very easily done if you consider each 4KB block as
>>>  a page beginning from location 0 upto whatever you've found to be the
>>>  highest memory available either thru BIOS or GRUB.
>>>
>>>  Remember that while setting up your PTEs and PDE every address is a
>>>  physical one. So if you thought that your kernel would be linked initially
>>>  to a higher VA since you would remap it to a lower memory physically then
>>>  that would be WRONG!. Without PTEs and PDEs installed don't do that!.
>>>
>>>  Why would you want it? Well for a simple reason, when your kernel starts
>>>  to boot there's no translator,(No PTEs/PDEs and the Paging Enabled bit of
>>>  processor is also cleared AFAIK just after the BIOS is done), yet since
>>>  you've not enabled your processor for that but you'll be doing that in a
>>>  moment.
>>>
>>>  So let's say you made your kernel to be linked to higher VA like 3Gigs.
>>>  Now the addresses would be generated beginning 3Gigs however you still
>>>  don't have the Page tables installed since your kernel just started. So in
>>>  that case the address is the physical address. And if you've not loaded
>>>  your kernel beginning 3Gigs then it would definitely come crashing down.
>>>
>>>  To avoid the crash in case you made your kernel to link to higher half of
>>>  the memory, you can use GDT trick since segmentation is always on and you
>>>  can make the overflow of the address addition to translate to a lower
>>>  physical memory even if paging is not enabled yet. Thus it is possible to
>>>  load the kernel at lower memory addresses while the linkage would be for
>>>  higher VMA. And once your PTEs/PGD are enabled then you can use those
>>>  instead of the GDT trick.
>>>
>>>  Here's a link to thathttp://wiki.osdev.org/Higher_Half_With_GDT
>>>
>>>>  Thanks
>>>>  Vaibhav Jain
>>  Hi,
>>
>>  Thanks for replying but I am still confused. I continued reading about this
>>  thing and what
>>  I have understood is the following :
>>  After the kernel executes the instruction to enable paging the instruction
>>  pointer will contain the
>>  address of the next instruction which will now be treated as a virtual
>>  address. So for the next instruction to be executed
>>  the page table should map this address to itself.
In this moment, the segment transition is still on, and physical address
is got from

Virtual Address ------------------>  Linear Address
----------------------->  Physical Address.
                 Segment                   Paging

Without identical mapping in LA[0]. PA = VA - 2* (kernel base address). With
identical mapping in LA[0], PA = VA - (kernel base address). It is just
what we
want. And with this mapping, we will disable the Segment transition by
setting
new segment offset to 0. After that, the PDE and PTE we seted for linear
address works and we don't need the identical mapping anymore.

Hope it heles.
>>  Please correct me if I am wrong.
>>  I am confused by the point about linking  the kernel to higher address.
>>  Could you please put that in a step by step manner
>>  to make it clear what  happens before paging is enabled and what happens
>>  after that.
>>  Also, please explain at what point during the execution of kernel code are
>>  the identity-mapped addresses no longer useful ?
>>
>>
>>
>>
>>  Thanks
>>  Vaibhav
>>  _______________________________________________
>>  Kernelnewbies mailing list
>>  Kernelnewbies at kernelnewbies.org
>>  http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
>  Hi
>
>  May be notes in [1] make it a bit clear ;)
>
>  [1]http://pankaj-techstuff.blogspot.com/2007/12/initialization-of-arm-mmu-in-linux.html
>
>  Best wishes
>  Vladimir Murzin
>
>  _______________________________________________
>  Kernelnewbies mailing list
>  Kernelnewbies at kernelnewbies.org
>  http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Regards,
Lewis Lau

identity mapped paging (Vaibhav Jain)

[Liu Zhiyou]

  ? 2012?04?17? 18:37, Vaibhav Jain ??:
>
>
> On Fri, Apr 13, 2012 at 2:15 AM, Vaibhav Jain <vjoss197@gmail.com 
> <mailto:vjoss197@gmail.com>> wrote:
>
>
>
>         > I am not clear about the use of identity mapped paging while
>         paging is
>         > being enabled by the operating system. Also I don't
>         understand at what
>         > point are the
>         > identity mappings no longer useful.According to this article
>         > http://geezer.osdevbrasil.net/osd/mem/index.htm#identity -
>         "The page
>         > table
>         > entries used to identity-map kernel memory can be deleted
>         once paging
>         > and
>         > virtual addresses are enabled." Can somebody please explain?
>         >
>
>         Identity mapping is when VA(Virt Address)=PA(Physical address).
>
>         So basically when you set up your page tables you need to make
>         sure they map identically. This is very easily done if you
>         consider each 4KB block as a page beginning from location 0
>         upto whatever you've found to be the highest memory available
>         either thru BIOS or GRUB.
>
>         Remember that while setting up your PTEs and PDE every address
>         is a physical one. So if you thought that your kernel would be
>         linked initially to a higher VA since you would remap it to a
>         lower memory physically then that would be WRONG!. Without
>         PTEs and PDEs installed don't do that!.
>
>         Why would you want it? Well for a simple reason, when your
>         kernel starts to boot there's no translator,(No PTEs/PDEs and
>         the Paging Enabled bit of processor is also cleared AFAIK just
>         after the BIOS is done), yet since you've not enabled your
>         processor for that but you'll be doing that in a moment.
>
>         So let's say you made your kernel to be linked to higher VA
>         like 3Gigs. Now the addresses would be generated beginning
>         3Gigs however you still don't have the Page tables installed
>         since your kernel just started. So in that case the address is
>         the physical address. And if you've not loaded your kernel
>         beginning 3Gigs then it would definitely come crashing down.
>
>         To avoid the crash in case you made your kernel to link to
>         higher half of the memory, you can use GDT trick since
>         segmentation is always on and you can make the overflow of the
>         address addition to translate to a lower physical memory even
>         if paging is not enabled yet. Thus it is possible to load the
>         kernel at lower memory addresses while the linkage would be
>         for higher VMA. And once your PTEs/PGD are enabled then you
>         can use those instead of the GDT trick.
>
>         Here's a link to that http://wiki.osdev.org/Higher_Half_With_GDT
>
>         > Thanks
>         > Vaibhav Jain
>
>
>
>     Hi,
>
>     Thanks for replying but I am still confused. I continued reading
>     about this thing and what
>     I have understood is the following :
>     After the kernel executes the instruction to enable paging the
>     instruction pointer will contain the
>     address of the next instruction which will now be treated as a
>     virtual address. So for the next instruction to be executed
>     the page table should map this address to itself.
>     Please correct me if I am wrong.
>     I am confused by the point about linking  the kernel to higher
>     address. Could you please put that in a step by step manner
>     to make it clear what  happens before paging is enabled and what
>     happens after that.
>     Also, please explain at what point during the execution of kernel
>     code are the identity-mapped addresses no longer useful ?
>
>
>
>
>     Thanks
>     Vaibhav
>
> Hi,
> I am somewhat understanding your point. But I have some other queries 
> now in my mind.
> If the kernel is linked to 3Gigs is there a way other than the GDT trick.?
> In fact I am wondering that if the kernel is linked to 3Gigs and Grub 
> loads it at 1MB physical, how will even the first instruction of kernel
> execute ?  I mean if all the address generated by kernel are above 3 
> Gigs and paging is not enabled how will it start
> running ?
> Thanks
> Vaibhav Jain
>
>
> _______________________________________________
> Kernelnewbies mailing list
> Kernelnewbies at kernelnewbies.org
> http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Vaibhav,
        Kernel want to be linked in 3Gigs address (Virtually, of 
course), but not every machine has so many memory. So at the very 
beginning GRUB just load it into a lower address like 1MB. After the 
loading, GRUB set eip/rip to the physical address where kernel loaded. 
 From then on, kernel code's running.
        And in that moment the GDT/Paging trick are not turned on so 
kernel will live with the world with only having physical address. But 
it's still Okay. In fact, because CPU simply add eip/rip rather than 
reseting it after one instrution is finished. So as long as the code not 
using instruction like ljmp(long jump, which will reset segment 
register(CS) and eip/rip), the code will run fine. So kernel setup a 
GDT, enable the segment transition, then using the ljmp to reset CS and 
eip/rip. From then on, kernel looks itself lives in the right VA.

Best,
Lewis Lau
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20120417/823f6aeb/attachment-0001.html 

identity mapped paging (Vaibhav Jain)

[Vaibhav Jain]

On Tue, Apr 17, 2012 at 3:46 AM, Pranay Kumar Srivastava <
Pranay.Shrivastava@hcl.com> wrote:

>
>
> > -----Original Message-----
> > From: Vaibhav Jain [mailto:vjoss197 at gmail.com]
> > Sent: Tuesday, April 17, 2012 4:07 PM
> > To: Pranay Kumar Srivastava
> > Cc: kernelnewbies at kernelnewbies.org
> > Subject: Re: identity mapped paging (Vaibhav Jain)
> >
> >
> > On Fri, Apr 13, 2012 at 2:15 AM, Vaibhav Jain <vjoss197@gmail.com>
> > wrote:
> >
> >
> > > I am not clear about the use of identity mapped paging while paging
> > is
> > > being enabled by the operating system. Also I don't understand at
> > what
> > > point are the
> > > identity mappings no longer useful.According to this article
> > > http://geezer.osdevbrasil.net/osd/mem/index.htm#identity - "The page
> > > table
> > > entries used to identity-map kernel memory can be deleted once paging
> > > and
> > > virtual addresses are enabled." Can somebody please explain?
> > >
> >
> > Identity mapping is when VA(Virt Address)=PA(Physical address).
> >
> > So basically when you set up your page tables you need to make sure
> > they map identically. This is very easily done if you consider each 4KB
> > block as a page beginning from location 0 upto whatever you've found to
> > be the highest memory available either thru BIOS or GRUB.
> >
> > Remember that while setting up your PTEs and PDE every address is a
> > physical one. So if you thought that your kernel would be linked
> > initially to a higher VA since you would remap it to a lower memory
> > physically then that would be WRONG!. Without PTEs and PDEs installed
> > don't do that!.
> >
> > Why would you want it? Well for a simple reason, when your kernel
> > starts to boot there's no translator,(No PTEs/PDEs and the Paging
> > Enabled bit of processor is also cleared AFAIK just after the BIOS is
> > done), yet since you've not enabled your processor for that but you'll
> > be doing that in a moment.
> >
> > So let's say you made your kernel to be linked to higher VA like 3Gigs.
> > Now the addresses would be generated beginning 3Gigs however you still
> > don't have the Page tables installed since your kernel just started. So
> > in that case the address is the physical address. And if you've not
> > loaded your kernel beginning 3Gigs then it would definitely come
> > crashing down.
> >
> > To avoid the crash in case you made your kernel to link to higher half
> > of the memory, you can use GDT trick since segmentation is always on
> > and you can make the overflow of the address addition to translate to a
> > lower physical memory even if paging is not enabled yet. Thus it is
> > possible to load the kernel at lower memory addresses while the linkage
> > would be for higher VMA. And once your PTEs/PGD are enabled then you
> > can use those instead of the GDT trick.
> >
> > Here's a link to that http://wiki.osdev.org/Higher_Half_With_GDT
> >
> > > Thanks
> > > Vaibhav Jain
> >
> > Hi,
> >
> > Thanks for replying but I am still confused. I continued reading about
> > this thing and what
> > I have understood is the following :
> > After the kernel executes the instruction to enable paging the
> > instruction pointer will contain the
> > address of the next instruction which will now be treated as a virtual
> > address. So for the next instruction to be executed
> > the page table should map this address to itself.
> > Please correct me if I am wrong.
> > I am confused by the point about linking  the kernel to higher address.
> > Could you please put that in a step by step manner
> > to make it clear what  happens before paging is enabled and what
> > happens after that.
> > Also, please explain at what point during the execution of kernel code
> > are the identity-mapped addresses no longer useful ?
> >
> >
> >
> >
> > Thanks
> > Vaibhav
> > Hi,
> >
> > I am somewhat understanding your point. But I have some other queries
> > now in my mind.
> >
> > If the kernel is linked to 3Gigs is there a way other than the GDT
> > trick.?
>
> Make your load address = VA when you link so you won't have to worry about
> doing the GDT trick.
>
> >
> > In fact I am wondering that if the kernel is linked to 3Gigs and Grub
> > loads it at 1MB physical, how will even the first instruction of kernel
> > execute ?  I mean if all the address generated by kernel are above 3
> > Gigs and paging is not enabled how will it start
> > running ?
>
> That's what the GDT trick is for. If you read the intel/amd processor
> manuals the segmentation is always on. So when the address get generated
> your segment's base address is still added to the generated address before
> it is put on wire. You can add a constant offset (in your GDT's base
> address part) to the generated address to get the address beginning from
> the load address of your kernel.
>
> I would suggest you make the higher half kernel later and try to first
> create some code that can fragment your available memory into pages and
> store this information so you'll know what all pages are there. Next would
> be to do identity mapping, since your kernel VMA=LMA in your linker script
> this would be easier to do.
>
> When you get that paging enabled you can move on to higher half kernel. I
> would suggest you to work on page replacement algos and virtual memory
> management code side by side for better integration with paging in later
> stages.
>
> Maybe you can post your code if you are allowed to then I can have a look
> at it.
>
> >
> > Thanks
> > Vaibhav Jain
>
>
> -----------------------------------------------------------------------------------------------------------------------------------------------------------------
>


Thanks for the explanation!!  Please confirm what I have concluded :
- If the kernel is linked to the same address it is loaded at ,
identiy-mapping is required so that all the addresses generated by the
kernel map to correct memory.

- If the kernel is NOT loaded at  the same address it is linked to (for
e.g. linked to 3 Gigs and loaded at 1MB) identity mapping is not required.

- if the kernel is linked AND loaded to 1MB, then to move to higher half
kernel two types of page mappings will be required -

a) identity-mapping of lower addresses
b) mapping all the virutal addresses above 3Gigs  to lower adresses



Actually I am going through a tutorial to write a kernel. In the tutorial
the linker script sets the location counter to 1MB and the code is working
fine.
But I looked at some other tutorials and osdev articles which mention about
higher half kernel and linking to 3Gigs and got confused.So I wanted to
be clear about the concepts before I make any changes to the code.


 Thanks
Vaibhav Jain
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20120417/bcb6f4ea/attachment-0001.html 

identity mapped paging (Vaibhav Jain)

[Pranay Kumar Srivastava]

> -----Original Message-----
> From: Vaibhav Jain [mailto:vjoss197 at gmail.com]
> Sent: Wednesday, April 18, 2012 3:49 AM
> To: Pranay Kumar Srivastava
> Cc: kernelnewbies at kernelnewbies.org
> Subject: Re: identity mapped paging (Vaibhav Jain)
>
> On Tue, Apr 17, 2012 at 3:46 AM, Pranay Kumar Srivastava
> <Pranay.Shrivastava@hcl.com> wrote:
>
>
> > -----Original Message-----
> > From: Vaibhav Jain [mailto:vjoss197 at gmail.com]
> > Sent: Tuesday, April 17, 2012 4:07 PM
> > To: Pranay Kumar Srivastava
> > Cc: kernelnewbies at kernelnewbies.org
> > Subject: Re: identity mapped paging (Vaibhav Jain)
> >
> >
> > On Fri, Apr 13, 2012 at 2:15 AM, Vaibhav Jain <vjoss197@gmail.com>
> > wrote:
> >
> >
> > > I am not clear about the use of identity mapped paging while paging
> > is
> > > being enabled by the operating system. Also I don't understand at
> > what
> > > point are the
> > > identity mappings no longer useful.According to this article
> > > http://geezer.osdevbrasil.net/osd/mem/index.htm#identity - "The
> page
> > > table
> > > entries used to identity-map kernel memory can be deleted once
> paging
> > > and
> > > virtual addresses are enabled." Can somebody please explain?
> > >
> >
> > Identity mapping is when VA(Virt Address)=PA(Physical address).
> >
> > So basically when you set up your page tables you need to make sure
> > they map identically. This is very easily done if you consider each
> 4KB
> > block as a page beginning from location 0 upto whatever you've found
> to
> > be the highest memory available either thru BIOS or GRUB.
> >
> > Remember that while setting up your PTEs and PDE every address is a
> > physical one. So if you thought that your kernel would be linked
> > initially to a higher VA since you would remap it to a lower memory
> > physically then that would be WRONG!. Without PTEs and PDEs installed
> > don't do that!.
> >
> > Why would you want it? Well for a simple reason, when your kernel
> > starts to boot there's no translator,(No PTEs/PDEs and the Paging
> > Enabled bit of processor is also cleared AFAIK just after the BIOS is
> > done), yet since you've not enabled your processor for that but
> you'll
> > be doing that in a moment.
> >
> > So let's say you made your kernel to be linked to higher VA like
> 3Gigs.
> > Now the addresses would be generated beginning 3Gigs however you
> still
> > don't have the Page tables installed since your kernel just started.
> So
> > in that case the address is the physical address. And if you've not
> > loaded your kernel beginning 3Gigs then it would definitely come
> > crashing down.
> >
> > To avoid the crash in case you made your kernel to link to higher
> half
> > of the memory, you can use GDT trick since segmentation is always on
> > and you can make the overflow of the address addition to translate to
> a
> > lower physical memory even if paging is not enabled yet. Thus it is
> > possible to load the kernel at lower memory addresses while the
> linkage
> > would be for higher VMA. And once your PTEs/PGD are enabled then you
> > can use those instead of the GDT trick.
> >
> > Here's a link to that http://wiki.osdev.org/Higher_Half_With_GDT
> >
> > > Thanks
> > > Vaibhav Jain
> >
> > Hi,
> >
> > Thanks for replying but I am still confused. I continued reading
> about
> > this thing and what
> > I have understood is the following :
> > After the kernel executes the instruction to enable paging the
> > instruction pointer will contain the
> > address of the next instruction which will now be treated as a
> virtual
> > address. So for the next instruction to be executed
> > the page table should map this address to itself.
> > Please correct me if I am wrong.
> > I am confused by the point about linking  the kernel to higher
> address.
> > Could you please put that in a step by step manner
> > to make it clear what  happens before paging is enabled and what
> > happens after that.
> > Also, please explain at what point during the execution of kernel
> code
> > are the identity-mapped addresses no longer useful ?
> >
> >
> >
> >
> > Thanks
> > Vaibhav
> > Hi,
> >
> > I am somewhat understanding your point. But I have some other queries
> > now in my mind.
> >
> > If the kernel is linked to 3Gigs is there a way other than the GDT
> > trick.?
> Make your load address = VA when you link so you won't have to worry
> about doing the GDT trick.
>
> >
> > In fact I am wondering that if the kernel is linked to 3Gigs and Grub
> > loads it at 1MB physical, how will even the first instruction of
> kernel
> > execute ?  I mean if all the address generated by kernel are above 3
> > Gigs and paging is not enabled how will it start
> > running ?
> That's what the GDT trick is for. If you read the intel/amd processor
> manuals the segmentation is always on. So when the address get
> generated your segment's base address is still added to the generated
> address before it is put on wire. You can add a constant offset (in
> your GDT's base address part) to the generated address to get the
> address beginning from the load address of your kernel.
>
> I would suggest you make the higher half kernel later and try to first
> create some code that can fragment your available memory into pages and
> store this information so you'll know what all pages are there. Next
> would be to do identity mapping, since your kernel VMA=LMA in your
> linker script this would be easier to do.
>
> When you get that paging enabled you can move on to higher half kernel.
> I would suggest you to work on page replacement algos and virtual
> memory management code side by side for better integration with paging
> in later stages.
>
> Maybe you can post your code if you are allowed to then I can have a
> look at it.
>
> >
> > Thanks
> > Vaibhav Jain
>
> -----------------------------------------------------------------------
> -----------------------------------------------------------------------
> -------------------
>
>
> Thanks for the explanation!!  Please confirm what I have concluded :
> - If the kernel is linked to the same address it is loaded at ,
> identiy-mapping is required so that all the addresses generated by the
> kernel map to correct memory.
>
> - If the kernel is NOT loaded at  the same address it is linked to (for
> e.g. linked to 3 Gigs and loaded at 1MB) identity mapping is not
> required.


Yes and No. Yes if you don't have paging enabled, NO if you will be enabling paging but you haven't mapped your physical
Memory yet to the addresses. Like I said before make a memory management routine before you enable paging, trust me you'll save a hell lot of time in later stages if you are really serious about making your kernel work.

>
> - if the kernel is linked AND loaded to 1MB, then to move to higher
> half kernel two types of page mappings will be required -
>
> a) identity-mapping of lower addresses
> b) mapping all the virutal addresses above 3Gigs  to lower adresses
>
Yes and No. Yes when initially you setup paging, the paging code would itself be in the process of being setup so most probably you would require identity paging here if you are not using GDT trick.
No when the rest of your kernel, i.e. after enabling paging, is being loaded such that it is able to work if its linked to higher VMA. This is since you already have got your page tables up and initialized.

What you can try here is split your kernel in two halves, one that just enables enough paging for you to work with initially. The rest of it you can even have as an executable and have code that loads it properly.

>
>
> Actually I am going through a tutorial to write a kernel. In the
> tutorial the linker script sets the location counter to 1MB and the
> code is working fine.


> But I looked at some other tutorials and osdev articles which mention
> about higher half kernel and linking to 3Gigs and got confused.So I
> wanted to
> be clear about the concepts before I make any changes to the code.
>
Osdev articles are very good but you should understand linker scripts and try identity paging first. Make your code kernel code page fault by writing in some memory location not mapped and check if you are able to handle that. After that move to higher half kernel, it would be best though to split your kernel at that point since it would become too big to load I guess by grub, but maybe your disk driver can do that?

>
>  Thanks
> Vaibhav Jain
>
>

::DISCLAIMER::
-----------------------------------------------------------------------------------------------------------------------

The contents of this e-mail and any attachment(s) are confidential and intended for the named recipient(s) only.
It shall not attach any liability on the originator or HCL or its affiliates. Any views or opinions presented in
this email are solely those of the author and may not necessarily reflect the opinions of HCL or its affiliates.
Any form of reproduction, dissemination, copying, disclosure, modification, distribution and / or publication of
this message without the prior written consent of the author of this e-mail is strictly prohibited. If you have
received this email in error please delete it and notify the sender immediately. Before opening any mail and
attachments please check them for viruses and defect.

-----------------------------------------------------------------------------------------------------------------------