[Bug 32277] New: overflow in calculate_miptree_layout_r300

[Bug 32277] New: overflow in calculate_miptree_layout_r300

[bugzilla-daemon]

https://bugs.freedesktop.org/show_bug.cgi?id=32277

           Summary: overflow in calculate_miptree_layout_r300
           Product: Mesa
           Version: git
          Platform: Other
        OS/Version: All
            Status: NEW
          Severity: major
          Priority: medium
         Component: Drivers/DRI/R600
        AssignedTo: dri-devel@lists.freedesktop.org
        ReportedBy: prahal@yahoo.com


Created an attachment (id=40964)
 View: https://bugs.freedesktop.org/attachment.cgi?id=40964
 Review: https://bugs.freedesktop.org/review?bug=32277&attachment=40964

Fix for the levels array overflow

extremetuxracer 0.5 beta2 shows up a bug in r600c. 

The levels array in radeon_mipmap_tree structure is defined with a size of
RADEON_MIPTREE_MAX_TEXTURE (ie 13). Though in radeon_try_alloc_miptree the size
of numLevels can overflow this size.
Then in calculate_miptree_layout_r300 the loop write out of the array.
Which leads to calloc failure in bo_open from radeon_gem_bo due to corrupted
memory.

This patch fixes this by setting the numLevels max to
RADEON_MIPTREE_MAX_TEXTURE.

-- 
Configure bugmail: https://bugs.freedesktop.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

[Bug 32277] overflow in calculate_miptree_layout_r300

[bugzilla-daemon]

https://bugs.freedesktop.org/show_bug.cgi?id=32277

--- Comment #1 from Alban Browaeys <prahal@yahoo.com> 2010-12-09 14:11:16 PST ---
Created an attachment (id=40965)
 --> (https://bugs.freedesktop.org/attachment.cgi?id=40965)
backtrace of the calloc failure in radeon drm due to this previous corruption

-- 
Configure bugmail: https://bugs.freedesktop.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

[Bug 32277] overflow in calculate_miptree_layout_r300

[bugzilla-daemon]

https://bugs.freedesktop.org/show_bug.cgi?id=32277

--- Comment #2 from Alban Browaeys <prahal@yahoo.com> 2010-12-09 14:14:53 PST ---
Created an attachment (id=40967)
 --> (https://bugs.freedesktop.org/attachment.cgi?id=40967)
valgrind output from such an etracer run which shows the overflow

the overflow happens in  in radeon_mipmap_tree.c:195 ie
calculate_miptree_layout_r300 and has origin radeon_try_alloc_miptree .

-- 
Configure bugmail: https://bugs.freedesktop.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

[Bug 32277] overflow in calculate_miptree_layout_r300

[bugzilla-daemon]

https://bugs.freedesktop.org/show_bug.cgi?id=32277

--- Comment #3 from Alex Deucher <agd5f@yahoo.com> 2010-12-12 15:55:31 PST ---
This should be fixed in:
http://cgit.freedesktop.org/mesa/mesa/commit/?id=fd543e1f9506fe41e6e9e78aebbe0bca01df055c

-- 
Configure bugmail: https://bugs.freedesktop.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

[Bug 32277] overflow in calculate_miptree_layout_r300

[bugzilla-daemon]

https://bugs.freedesktop.org/show_bug.cgi?id=32277

Alban Browaeys <prahal@yahoo.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
         Resolution|                            |FIXED

--- Comment #4 from Alban Browaeys <prahal@yahoo.com> 2010-12-20 07:06:40 PST ---
User reported the issue as fixed. Can be closed. Thank you

-- 
Configure bugmail: https://bugs.freedesktop.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.