From: Hubert Kario <hubert@kario.pl>
To: linux-mm@kvack.org
Subject: Re: Big kernel dynamic memory usage (non-cache)
Date: Wed, 23 Jan 2013 11:50:55 +0100 [thread overview]
Message-ID: <4696903.iLjIsJ2Mj0@bursa22> (raw)
In-Reply-To: <2233004.c1QkIFnMrK@bursa22>
Anyone?
Is this not the correct mailing list for this problem?
If yes, are there any tools to help diagnose it? Where should I begin?
I can reproduce this problem regularly, but it takes quite a bit of time, so
I don't expect other people to diagnose and fix it.
Regards,
Hubert
On Tuesday 22 of January 2013 02:50:40 Hubert Kario wrote:
> Hi!
>
> In short, how to get detailed information what is the "noncache" column in
> "kernel dynamic memory" in smem -w output?
>
> # smem -wtk
> Area Used Cache Noncache
> firmware/hardware 0 0 0
> kernel image 0 0 0
> kernel dynamic memory 1.5G 637.7M 853.5M
> userspace memory 2.0G 142.7M 1.9G
> free memory 383.5M 383.5M 0
> ----------------------------------------------------------
> 3.9G 1.1G 2.7G
>
> Little background info:
>
> I'm experiencing a weird problem with the kernel, where it does allocate
> lots of memory, but I can't find what it is used for.
> After the system has been running for few days (around a week usually) it
> starts to swap quite heavily (if I don't close the biggest applications
> quickly to recover I usually have to reset it as even remote ssh access
> doesn't work, it responds to pings so it's not a hard lock up, Magic SysRq
> doesn't restart it though). Thing is, it's enough to log-out from the X
> environment and back in to make it go back to normal (where the noncache
> memory is less than 200MiB).
>
> It's a workstation, with KMail, Opera and the usual background KDE
> services running in the back (with file indexing turned off). Any other
> processes are either short lived or not using any big amounts (>5MiB) of
> memory. Archlinux, so the packages (both kernel and userspace) are either
> vanilla or with minimal patches.
>
> The sum of USS of all processes in the system, as reported by smem
> (the sum of RSS from `ps` is only 10-15% larger), during all that time
> oscillates around 2GiB and basically never reaches 3GiB. I have tried
> running with zram and it made the situation a bit better, but it
> sometimes still locks up (because it starts swapping so hard).
>
> As restarting X helps, it led me to believe it may be a regression from
> this bug: https://bugzilla.redhat.com/show_bug.cgi?id=615505
> but it doesn't look like it to me:
>
> # cat /sys/kernel/debug/dri/0/gem_names
> name size handles refcount
> 1 7680000 1 3
> 2 15564800 1 3
>
> Some statistics and information on the system follow.
>
> Regards,
> Hubert Kario
>
> # uname -a
> Linux bursa22 3.6.11-1-ARCH #1 SMP PREEMPT Tue Dec 18 08:57:15 CET 2012
> x86_64 GNU/Linux
>
> # free -k
> total used free shared buffers cached
> Mem: 4051956 3598176 453780 0 30264 646828
> -/+ buffers/cache: 2921084 1130872
> Swap: 4051952 292524 3759428
>
> # lsmod
> Module Size Used by
> vboxdrv 1823470 0
> iptable_filter 1457 1
> iptable_mangle 1585 1
> xt_tcpudp 2472 14
> xt_connmark 1910 12
> xt_mark 1214 13
> iptable_nat 4025 1
> nf_nat 15371 1 iptable_nat
> nf_conntrack_ipv4 7799 15 nf_nat,iptable_nat
> nf_defrag_ipv4 1340 1 nf_conntrack_ipv4
> nf_conntrack 64101 4
> nf_nat,xt_connmark,iptable_nat,nf_conntrack_ipv4 ip_tables
> 16947 3 iptable_filter,iptable_mangle,iptable_nat x_tables
> 17000 7
> xt_mark,ip_tables,xt_tcpudp,iptable_filter,xt_connmark,iptable_mangle,ipt
> able_nat tun 16593 2
> zram 9460 2
> xfs 748208 1
> snd_hda_codec_hdmi 24529 1
> fuse 69213 5
> btrfs 718583 2
> coretemp 6071 0
> kvm_intel 124718 0
> crc32c 1769 1
> libcrc32c 1003 1 btrfs
> zlib_deflate 20437 1 btrfs
> kvm 374014 1 kvm_intel
> radeon 859687 2
> iTCO_wdt 5256 0
> iTCO_vendor_support 1930 1 iTCO_wdt
> gpio_ich 4064 0
> snd_hda_codec_realtek 61420 1
> ttm 64385 1 radeon
> drm_kms_helper 32878 1 radeon
> drm 218903 4 ttm,drm_kms_helper,radeon
> r8169 56872 0
> i2c_algo_bit 5392 1 radeon
> snd_hda_intel 26181 4
> mii 4092 1 r8169
> snd_hda_codec 98034 3
> snd_hda_codec_realtek,snd_hda_codec_hdmi,snd_hda_intel snd_hwdep
> 6429 1 snd_hda_codec
> snd_pcm 75735 3
> snd_hda_codec_hdmi,snd_hda_codec,snd_hda_intel snd_page_alloc
> 7218 2 snd_pcm,snd_hda_intel
> snd_timer 18935 1 snd_pcm
> acpi_cpufreq 5934 0
> snd 60189 15
> snd_hda_codec_realtek,snd_hwdep,snd_timer,snd_hda_codec_hdmi,snd_pcm,snd_
> hda_codec,snd_hda_intel mperf 1300 1 acpi_cpufreq
> intel_agp 10745 0
> i2c_i801 9572 0
> soundcore 5443 1 snd
> intel_gtt 15660 1 intel_agp
> microcode 12346 0
> i2c_core 20708 5
> drm,i2c_i801,drm_kms_helper,i2c_algo_bit,radeon lpc_ich
> 10610 0
> pcspkr 1900 0
> processor 26856 3 acpi_cpufreq
> button 4663 0
> evdev 10267 7
> ext4 440435 4
> crc16 1360 1 ext4
> jbd2 78802 1 ext4
> mbcache 6027 1 ext4
> dm_mod 72106 25
> raid456 55308 1
> async_raid6_recov 5375 1 raid456
> async_memcpy 1367 2 raid456,async_raid6_recov
> async_pq 3261 2 raid456,async_raid6_recov
> async_xor 2394 3 async_pq,raid456,async_raid6_recov
> xor 8866 1 async_xor
> async_tx 1903 5
> async_pq,raid456,async_xor,async_memcpy,async_raid6_recov raid6_pq
> 86983 2 async_pq,async_raid6_recov
> raid10 40365 1
> md_mod 104017 3 raid456,raid10
> hid_generic 1114 0
> usbhid 37036 0
> hid 85974 2 hid_generic,usbhid
> sr_mod 14824 0
> cdrom 35521 1 sr_mod
> sd_mod 29560 27
> ata_generic 3339 0
> uhci_hcd 23437 0
> pata_acpi 3452 0
> sata_via 8124 5
> ahci 21361 12
> libahci 20024 1 ahci
> pata_it8213 3587 1
> firewire_ohci 31942 0
> libata 167757 6
> ahci,pata_acpi,sata_via,libahci,ata_generic,pata_it8213 scsi_mod
> 133434 3 libata,sd_mod,sr_mod
> firewire_core 53174 1 firewire_ohci
> crc_itu_t 1364 1 firewire_core
> ehci_hcd 41849 0
> usbcore 150472 3 uhci_hcd,ehci_hcd,usbhid
> usb_common 955 1 usbcore
>
> # smem -kt | sed -n '1p;$p'
> PID User Command Swap USS PSS
> RSS 110 6 281.6M 2.0G 2.0G
> 2.3G
>
> # cat /proc/meminfo
> MemTotal: 4051956 kB
> MemFree: 452512 kB
> Buffers: 29784 kB
> Cached: 648796 kB
> SwapCached: 144308 kB
> Active: 1555176 kB
> Inactive: 1175416 kB
> Active(anon): 1238968 kB
> Inactive(anon): 868836 kB
> Active(file): 316208 kB
> Inactive(file): 306580 kB
> Unevictable: 7720 kB
> Mlocked: 7720 kB
> SwapTotal: 4051952 kB
> SwapFree: 3759416 kB
> Dirty: 372 kB
> Writeback: 0 kB
> AnonPages: 1982576 kB
> Mapped: 148724 kB
> Shmem: 52248 kB
> Slab: 185580 kB
> SReclaimable: 69316 kB
> SUnreclaim: 116264 kB
> KernelStack: 3320 kB
> PageTables: 43996 kB
> NFS_Unstable: 0 kB
> Bounce: 0 kB
> WritebackTmp: 0 kB
> CommitLimit: 6077928 kB
> Committed_AS: 3751688 kB
> VmallocTotal: 34359738367 kB
> VmallocUsed: 139780 kB
> VmallocChunk: 34359570976 kB
> HardwareCorrupted: 0 kB
> AnonHugePages: 442368 kB
> HugePages_Total: 0
> HugePages_Free: 0
> HugePages_Rsvd: 0
> HugePages_Surp: 0
> Hugepagesize: 2048 kB
> DirectMap4k: 2536128 kB
> DirectMap2M: 1656832 kB
>
> # cat /proc/vmstat
> nr_free_pages 113256
> nr_inactive_anon 217210
> nr_active_anon 309694
> nr_inactive_file 76546
> nr_active_file 79049
> nr_unevictable 1930
> nr_mlock 1930
> nr_anon_pages 385048
> nr_mapped 37179
> nr_file_pages 205623
> nr_dirty 24
> nr_writeback 0
> nr_slab_reclaimable 17329
> nr_slab_unreclaimable 29066
> nr_page_table_pages 10973
> nr_kernel_stack 413
> nr_unstable 0
> nr_bounce 0
> nr_vmscan_write 3103053
> nr_vmscan_immediate_reclaim 21226934
> nr_writeback_temp 0
> nr_isolated_anon 0
> nr_isolated_file 0
> nr_shmem 13062
> nr_dirtied 39768555
> nr_written 38668348
> numa_hit 1043570022
> numa_miss 0
> numa_foreign 0
> numa_interleave 7629
> numa_local 1043570022
> numa_other 0
> nr_anon_transparent_hugepages 216
> nr_dirty_threshold 76584
> nr_dirty_background_threshold 38292
> pgpgin 538657436
> pgpgout 172521740
> pswpin 350320
> pswpout 404062
> pgalloc_dma 2
> pgalloc_dma32 666912295
> pgalloc_normal 571432892
> pgalloc_movable 0
> pgfree 1243823469
> pgactivate 73410706
> pgdeactivate 72896276
> pgfault 544723643
> pgmajfault 2023827
> pgrefill_dma 0
> pgrefill_dma32 64507672
> pgrefill_normal 18650343
> pgrefill_movable 0
> pgsteal_kswapd_dma 0
> pgsteal_kswapd_dma32 3005794
> pgsteal_kswapd_normal 26891312
> pgsteal_kswapd_movable 0
> pgsteal_direct_dma 0
> pgsteal_direct_dma32 15653023
> pgsteal_direct_normal 42635574
> pgsteal_direct_movable 0
> pgscan_kswapd_dma 0
> pgscan_kswapd_dma32 3136881
> pgscan_kswapd_normal 32518914
> pgscan_kswapd_movable 0
> pgscan_direct_dma 0
> pgscan_direct_dma32 361983838
> pgscan_direct_normal 6556538987
> pgscan_direct_movable 0
> pgscan_direct_throttle 0
> zone_reclaim_failed 0
> pginodesteal 24512
> slabs_scanned 35039616
> kswapd_inodesteal 193648
> kswapd_low_wmark_hit_quickly 654504
> kswapd_high_wmark_hit_quickly 690974
> kswapd_skip_congestion_wait 2977
> pageoutrun 645618572
> allocstall 7647697
> pgrotated 285215
> compact_blocks_moved 23846881
> compact_pages_moved 4897380
> compact_pagemigrate_failed 48162348
> compact_stall 145117
> compact_fail 138243
> compact_success 6874
> htlb_buddy_alloc_success 0
> htlb_buddy_alloc_fail 0
> unevictable_pgs_culled 8443
> unevictable_pgs_scanned 0
> unevictable_pgs_rescued 7075596
> unevictable_pgs_mlocked 7082893
> unevictable_pgs_munlocked 7076462
> unevictable_pgs_cleared 0
> unevictable_pgs_stranded 0
> unevictable_pgs_mlockfreed 0
> thp_fault_alloc 15563
> thp_fault_fallback 59524
> thp_collapse_alloc 1356
> thp_collapse_alloc_failed 4821
> thp_split 513
>
> # cat /proc/swaps
> Filename Type Size Used
> Priority /dev/zram0 partition 2025976
> 146416 100 /dev/zram1 partition
> 2025976 146108 100
>
> # cat /sys/block/zram*/compr_data_size
> 47152674
> 47090766
--
Hubert Kario
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
prev parent reply other threads:[~2013-01-23 10:50 UTC|newest]
Thread overview: 2+ messages / expand[flat|nested] mbox.gz Atom feed top
2013-01-22 1:50 Big kernel dynamic memory usage (non-cache) Hubert Kario
2013-01-23 10:50 ` Hubert Kario [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=4696903.iLjIsJ2Mj0@bursa22 \
--to=hubert@kario.pl \
--cc=linux-mm@kvack.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).