[issue1705] shmat on x86_64 causes SIGSEGV in x11/gtk2

["David Shao \(via DragonFly issue tracker\)" <sinknull@xxxxxxxxxxxxxxxxxxxxx>]

New submission from David Shao <davshao@gmail.com>:

To reproduce, on an x86_64 machine (tested on a Shuttle SG45H7 and an Intel Core
2 Duo Macbook 2.1), from a current DragonFly v2.5.1 1061.ge9fe2 X86_64_GENERIC
build, install modular xorg and gtk+-2.18.9 from current pkgsrc, and run
git-demo which SIGSEVs.  No such SIGSEGV occurs on an i386 machine.

Attached is a gdb backtrace.  Of interest are gtk2 GdkImage, in this case,
parameter image, a struct that has a field mem that is if shm is enabled on
x86_64 produces an

 invalid address 0x8400000:      Cannot access memory at address 0x8400000

Actually image_info also has a field that points to a list static_image of
GdkImage's.

----------
files: gtk-demo_debug.txt
keyword: kernel, pkgsrc
messages: 8308
nosy: davshao
status: unread
title: shmat on x86_64 causes SIGSEGV in x11/gtk2

_____________________________________________________
DragonFly issue tracker <bugs@lists.dragonflybsd.org>
<http://bugs.dragonflybsd.org/issue1705>
_____________________________________________________

GNU gdb (GDB) 7.0
Copyright (C) 2009 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-dragonfly".
For bug reporting instructions, please see:
<http://bugs.dragonflybsd.org/>...
Reading symbols from /usr/pkg/bin/gtk-demo...done.
(gdb) start
Temporary breakpoint 1 at 0x422873: file main.c, line 941.
Starting program: /usr/pkg/bin/gtk-demo 

Temporary breakpoint 1, main (argc=1, argv=0x7ffffffff890) at main.c:941
941	  if (g_file_test ("../../gdk-pixbuf/libpixbufloader-pnm.la",
(gdb) continue
Continuing.

Program received signal SIGSEGV, Segmentation fault.
0x00000008007c6577 in gdk_rgb_convert_565 (image_info=0x8039c1b90, image=0x8005f82a0, x0=0, y0=0, width=107, height=64, 
    buf=0x80428d000 '\377' <repeats 200 times>..., rowstride=768, x_align=0, y_align=0, cmap=0x0) at gdkrgb.c:1522
1522		      ((guint32 *)obptr)[0] =
(gdb) bt
#0  0x00000008007c6577 in gdk_rgb_convert_565 (image_info=0x8039c1b90, image=0x8005f82a0, x0=0, y0=0, width=107, height=64, 
    buf=0x80428d000 '\377' <repeats 200 times>..., rowstride=768, x_align=0, y_align=0, cmap=0x0) at gdkrgb.c:1522
#1  0x00000008007c8f3a in gdk_rgb_convert_32_generic (image_info=0x8039c1b90, image=0x8005f82a0, x0=0, y0=0, width=107, 
    height=64, buf=0x803c38000 "\377\377\377", rowstride=428, x_align=0, y_align=0, cmap=0x0) at gdkrgb.c:2895
#2  0x00000008007c9cc9 in gdk_draw_rgb_image_core (image_info=0x8039c1b90, drawable=0x8005d8780, gc=0x8005f8830, x=0, y=0, 
    width=107, height=140, buf=0x803c38000 "\377\377\377", pixstride=4, rowstride=428, 
    conv=0x8007c8ea4 <gdk_rgb_convert_32_generic>, cmap=0x0, xdith=0, ydith=0) at gdkrgb.c:3330
#3  0x00000008007ca0f6 in IA__gdk_draw_rgb_32_image (drawable=0x8005d8780, gc=0x8005f8830, x=0, y=0, width=107, height=140, 
    dith=GDK_RGB_DITHER_NORMAL, buf=0x803c38000 "\377\377\377", rowstride=428) at gdkrgb.c:3442
#4  0x00000008007bdf02 in IA__gdk_pixbuf_render_pixmap_and_mask_for_colormap (pixbuf=0x8006209e0, colormap=0x8005f1820, 
    pixmap_return=0x8005d8818, mask_return=0x8005d8820, alpha_threshold=128) at gdkpixbuf-render.c:315
#5  0x0000000800c4716f in get_pixmap_and_mask (window=0x80056d360, parent_info=0x0, is_default_list=1, icon_list=0x8005d0b60, 
    pmap_return=0x8005d8818, mask_return=0x8005d8820) at gtkwindow.c:3142
#6  0x0000000800c475b2 in gtk_window_realize_icon (window=0x80056d250) at gtkwindow.c:3278
#7  0x0000000800c49b0c in gtk_window_realize (widget=0x80056d250) at gtkwindow.c:4854
#8  0x0000000802fe86f9 in IA__g_cclosure_marshal_VOID__VOID (closure=0x8039909a0, return_value=0x0, n_param_values=1, 
    param_values=0x8005d64a0, invocation_hint=0x7fffffffe970, marshal_data=0x800c493d9) at gmarshal.c:77
#9  0x0000000802fce331 in g_type_class_meta_marshal (closure=0x8039909a0, return_value=0x0, n_param_values=1, 
    param_values=0x8005d64a0, invocation_hint=0x7fffffffe970, marshal_data=0xe0) at gclosure.c:878
#10 0x0000000802fce021 in IA__g_closure_invoke (closure=0x8039909a0, return_value=0x0, n_param_values=1, 
    param_values=0x8005d64a0, invocation_hint=0x7fffffffe970) at gclosure.c:767
#11 0x0000000802fe75dc in signal_emit_unlocked_R (node=0x803971d80, detail=0, instance=0x80056d250, emission_return=0x0, 
    instance_and_params=0x8005d64a0) at gsignal.c:3177
#12 0x0000000802fe67a4 in IA__g_signal_emit_valist (instance=0x80056d250, signal_id=16, detail=0, var_args=0x7fffffffed00)
    at gsignal.c:2980
#13 0x0000000802fe6d8e in IA__g_signal_emit (instance=0x80056d250, signal_id=16, detail=0) at gsignal.c:3037
#14 0x0000000800c324ac in IA__gtk_widget_realize (widget=0x80056d250) at gtkwidget.c:3328
#15 0x0000000800c48e6f in gtk_window_show (widget=0x80056d250) at gtkwindow.c:4488
#16 0x0000000802fe86f9 in IA__g_cclosure_marshal_VOID__VOID (closure=0x803990920, return_value=0x0, n_param_values=1, 
    param_values=0x8005d6880, invocation_hint=0x7ffffffff2c0, marshal_data=0x800c48d48) at gmarshal.c:77
#17 0x0000000802fce331 in g_type_class_meta_marshal (closure=0x803990920, return_value=0x0, n_param_values=1, 
    param_values=0x8005d6880, invocation_hint=0x7ffffffff2c0, marshal_data=0xb0) at gclosure.c:878
#18 0x0000000802fce021 in IA__g_closure_invoke (closure=0x803990920, return_value=0x0, n_param_values=1, 
    param_values=0x8005d6880, invocation_hint=0x7ffffffff2c0) at gclosure.c:767
#19 0x0000000802fe75dc in signal_emit_unlocked_R (node=0x803971c40, detail=0, instance=0x80056d250, emission_return=0x0, 
    instance_and_params=0x8005d6880) at gsignal.c:3177
#20 0x0000000802fe67a4 in IA__g_signal_emit_valist (instance=0x80056d250, signal_id=12, detail=0, var_args=0x7ffffffff650)
    at gsignal.c:2980
#21 0x0000000802fe6d8e in IA__g_signal_emit (instance=0x80056d250, signal_id=12, detail=0) at gsignal.c:3037
#22 0x0000000800c31a6c in IA__gtk_widget_show (widget=0x80056d250) at gtkwidget.c:3012
#23 0x0000000800a2bc2a in gtk_container_show_all (widget=0x80056d250) at gtkcontainer.c:2595
#24 0x0000000800c31f11 in IA__gtk_widget_show_all (widget=0x80056d250) at gtkwidget.c:3160
#25 0x0000000000422b46 in main (argc=1, argv=0x7ffffffff890) at main.c:1005
(gdb) x image->mem
0x8400000:	Cannot access memory at address 0x8400000
(gdb) ptype image
type = struct _GdkImage {
    GObject parent_instance;
    GdkImageType type;
    GdkVisual *visual;
    GdkByteOrder byte_order;
    gint width;
    gint height;
    guint16 depth;
    guint16 bpp;
    guint16 bpl;
    guint16 bits_per_pixel;
    gpointer mem;
    GdkColormap *colormap;
    gpointer windowing_data;
} *
(gdb) print image
$1 = (GdkImage *) 0x8005f82a0
(gdb) quit
A debugging session is active.

	Inferior 1 [process 47176] will be killed.

Quit anyway? (y or n)