Opened 10 years ago

Closed 10 years ago

Last modified 3 years ago

#613 closed defect (fixed)

unittests in CONVERTALN crash when code gets optimized

Reported by: westram Owned by: westram
Priority: critical Milestone: arb7.0
Component: no idea Version: SVN
Keywords: Cc:

Description

Example:

  • compile arb with gcc 4.9.0 and DEBUG=0 in config.makefile
  • run unittests ⇒ crashes in CONVERTALN (with corrupted heap)

Notes:

  • happens with -O1 up to -O4
  • happens with gcc 4.9.1 as well
  • with gcc 4.8.2 and 4.8.3 test does not crash, but valgrind reports errors (revert helpless workaround [12992] to reproduce)
  • using AddressSanitizer with 4.9.1 reports a double free

Change History (8)

comment:1 Changed 10 years ago by westram

  • Status changed from new to _started

Already applied patches:

comment:2 follow-up: Changed 10 years ago by epruesse

Possibly relevant:

​https://www.sourceware.org/ml/binutils/2014-09/msg00175.html

I added a "-Wl,-nohinhibit-exec" to the Makefile so that I could get binaries to look at, having no cutting edge toolchain at hand and needing the Ubuntu/Debian? build systems for this (ours is full, no more targets possible).

comment:3 in reply to: ↑ 2 Changed 10 years ago by westram

Replying to epruesse:

Possibly relevant: I added a "-Wl,-nohinhibit-exec" to the Makefile

same same

comment:4 Changed 10 years ago by westram

backtrace for 4.9.0: 

*** glibc detected *** tests.slow/test_CONVERTALN_CONVERTALN_a: corrupted double-linked list: 0x00000000020060d0 ***
======= Backtrace: =========
/lib/libc.so.6(+0x788d6)[0x2b70782e28d6]
/lib/libc.so.6(+0x7d6b4)[0x2b70782e76b4]
/lib/libc.so.6(__libc_malloc+0x6e)[0x2b70782e78ae]
/opt/gcc-4.9.0/lib64/libstdc++.so.6(_Znwm+0x18)[0x2b7077b1d898]
/opt/gcc-4.9.0/lib64/libstdc++.so.6(_ZNSs4_Rep9_S_createEmmRKSaIcE+0x59)[0x2b7077b82639]
/opt/gcc-4.9.0/lib64/libstdc++.so.6(_ZNSs12_S_constructIPKcEEPcT_S3_RKSaIcESt20forward_iterator_tag+0x35)[0x2b7077b83ef5]
/opt/gcc-4.9.0/lib64/libstdc++.so.6(_ZNSsC1EPKcmRKSaIcE+0x1c)[0x2b7077b83f8c]
/home/ralf/ARB-bilbo/ARB.gcc.490.NDEBUG/lib/libCORE.so(_ZN18BufferedFileReader14getLine_internERSs+0xa4)[0x2b707750c5f4]
tests.slow/test_CONVERTALN_CONVERTALN_a(_ZN6Reader4readEv+0x118)[0x439758]
tests.slow/test_CONVERTALN_CONVERTALN_a[0x42c6ee]
tests.slow/test_CONVERTALN_CONVERTALN_a(_ZN13GenbankParser19parse_keyed_sectionEPKc+0x1b4)[0x42d974]
tests.slow/test_CONVERTALN_CONVERTALN_a(_ZN13GenbankReader14read_one_entryER3Seq+0x4f9)[0x42ede9]
tests.slow/test_CONVERTALN_CONVERTALN_a(_Z7convertRK13FormattedFileS1_+0x5d1)[0x4100e1]
tests.slow/test_CONVERTALN_CONVERTALN_a[0x426136]
tests.slow/test_CONVERTALN_CONVERTALN_a[0x42668e]
tests.slow/test_CONVERTALN_CONVERTALN_a(_Z19TEST_SLOW_converterv+0xce4)[0x429b94]
tests.slow/test_CONVERTALN_CONVERTALN_a[0x40c611]
tests.slow/test_CONVERTALN_CONVERTALN_a(_Z15execute_guardedPFvvEPllb+0x20)[0x40cf30]
tests.slow/test_CONVERTALN_CONVERTALN_a(_ZN12SimpleTester7performEm+0x1e6)[0x40d316]
tests.slow/test_CONVERTALN_CONVERTALN_a(_ZN12SimpleTester11perform_allEv+0xab)[0x40d9db]
tests.slow/test_CONVERTALN_CONVERTALN_a(_ZN10UnitTesterC1EPKcPK15UnitTest_simpleimS4_+0x45c)[0x40deac]
tests.slow/test_CONVERTALN_CONVERTALN_a[0x40c248]
/lib/libc.so.6(__libc_start_main+0xfd)[0x2b7078288c8d]
tests.slow/test_CONVERTALN_CONVERTALN_a[0x40c0dd]

valgrind report for gcc 4.8.3: 

(    ) ==26679== Invalid read of size 4
(note)  ==26679==    at 0x4253CB: gtoe(GenBank const&, Embl&, Seq const&) (in /home/ralf/ARB-bilbo/ARB.gcc.483.NDEBUG/UNIT_TESTER/tests.slow/test_CONVERTALN_CONVERTALN_a)
(note)  ==26679==    by 0x40FD26: ConvertibleData::to_embl(Seq const&) const (in /home/ralf/ARB-bilbo/ARB.gcc.483.NDEBUG/UNIT_TESTER/tests.slow/test_CONVERTALN_CONVERTALN_a)
(note)  ==26679==    by 0x410879: convert(FormattedFile const&, FormattedFile const&) (in /home/ralf/ARB-bilbo/ARB.gcc.483.NDEBUG/UNIT_TESTER/tests.slow/test_CONVERTALN_CONVERTALN_a)
(note)  ==26679==    by 0x426078: test_convert(char const*, char const*, Format, Format) (in /home/ralf/ARB-bilbo/ARB.gcc.483.NDEBUG/UNIT_TESTER/tests.slow/test_CONVERTALN_CONVERTALN_a)
(note)  ==26679==    by 0x42648C: test_convert_by_format_num(int, int) (in /home/ralf/ARB-bilbo/ARB.gcc.483.NDEBUG/UNIT_TESTER/tests.slow/test_CONVERTALN_CONVERTALN_a)
(note)  ==26679==    by 0x429498: TEST_SLOW_converter() (in /home/ralf/ARB-bilbo/ARB.gcc.483.NDEBUG/UNIT_TESTER/tests.slow/test_CONVERTALN_CONVERTALN_a)
(note)  ==26679==    by 0x40CFE0: execute_guarded_ClientCode(void (*)(), long*) (in /home/ralf/ARB-bilbo/ARB.gcc.483.NDEBUG/UNIT_TESTER/tests.slow/test_CONVERTALN_CONVERTALN_a)
(note)  ==26679==    by 0x40D82F: execute_guarded(void (*)(), long*, long, bool) (in /home/ralf/ARB-bilbo/ARB.gcc.483.NDEBUG/UNIT_TESTER/tests.slow/test_CONVERTALN_CONVERTALN_a)
(note)  ==26679==    by 0x40DC0D: SimpleTester::perform(unsigned long) (in /home/ralf/ARB-bilbo/ARB.gcc.483.NDEBUG/UNIT_TESTER/tests.slow/test_CONVERTALN_CONVERTALN_a)
(note)  ==26679==    by 0x40E24C: SimpleTester::perform_all() (in /home/ralf/ARB-bilbo/ARB.gcc.483.NDEBUG/UNIT_TESTER/tests.slow/test_CONVERTALN_CONVERTALN_a)
(note)  ==26679==    by 0x40E6FE: UnitTester::UnitTester(char const*, UnitTest_simple const*, int, unsigned long, UnitTest_simple const*) (in /home/ralf/ARB-bilbo/ARB.gcc.483.NDEBUG/UNIT_TESTER/tests.slow/test_CONVERTALN_CONVERTALN_a)
(note)  ==26679==    by 0x40CCB7: main (in /home/ralf/ARB-bilbo/ARB.gcc.483.NDEBUG/UNIT_TESTER/tests.slow/test_CONVERTALN_CONVERTALN_a)
(note) ==26679==    Invalid read of size 4 by 0x5F89C8C: __libc_start_main  (in unavailable file libc-start.c line 226)
(note)  ==26679==    by 0x40CB60: ??? (in /home/ralf/ARB-bilbo/ARB.gcc.483.NDEBUG/UNIT_TESTER/tests.slow/test_CONVERTALN_CONVERTALN_a)
(    ) ==26679==    by 0x7FEFFF647: ???
(    ) ==26679==    by 0x1B: ???
(    ) ==26679==  Address 0x65c7300 is 32 bytes inside a block of size 35 alloc'd
(note) ==26679==    ORIGIN: Address 0x65c7300 is 32 bytes inside a block of size 35 alloc'd at 0x4C274A8: malloc  (in unavailable file vg_replace_malloc.c line 236)
(note) ==26679==    ORIGIN: Address 0x65c7300 is 32 bytes inside a block of size 35 alloc'd by 0x5FEF241: strdup  (in unavailable file strdup.c line 43)
(note)  ==26679==    by 0x4253A0: gtoe(GenBank const&, Embl&, Seq const&) (in /home/ralf/ARB-bilbo/ARB.gcc.483.NDEBUG/UNIT_TESTER/tests.slow/test_CONVERTALN_CONVERTALN_a)
(note)  ==26679==    by 0x40FD26: ConvertibleData::to_embl(Seq const&) const (in /home/ralf/ARB-bilbo/ARB.gcc.483.NDEBUG/UNIT_TESTER/tests.slow/test_CONVERTALN_CONVERTALN_a)
(note)  ==26679==    by 0x410879: convert(FormattedFile const&, FormattedFile const&) (in /home/ralf/ARB-bilbo/ARB.gcc.483.NDEBUG/UNIT_TESTER/tests.slow/test_CONVERTALN_CONVERTALN_a)
(note)  ==26679==    by 0x426078: test_convert(char const*, char const*, Format, Format) (in /home/ralf/ARB-bilbo/ARB.gcc.483.NDEBUG/UNIT_TESTER/tests.slow/test_CONVERTALN_CONVERTALN_a)
(note)  ==26679==    by 0x42648C: test_convert_by_format_num(int, int) (in /home/ralf/ARB-bilbo/ARB.gcc.483.NDEBUG/UNIT_TESTER/tests.slow/test_CONVERTALN_CONVERTALN_a)
(note)  ==26679==    by 0x429498: TEST_SLOW_converter() (in /home/ralf/ARB-bilbo/ARB.gcc.483.NDEBUG/UNIT_TESTER/tests.slow/test_CONVERTALN_CONVERTALN_a)
(note)  ==26679==    by 0x40CFE0: execute_guarded_ClientCode(void (*)(), long*) (in /home/ralf/ARB-bilbo/ARB.gcc.483.NDEBUG/UNIT_TESTER/tests.slow/test_CONVERTALN_CONVERTALN_a)
(note)  ==26679==    by 0x40D82F: execute_guarded(void (*)(), long*, long, bool) (in /home/ralf/ARB-bilbo/ARB.gcc.483.NDEBUG/UNIT_TESTER/tests.slow/test_CONVERTALN_CONVERTALN_a)
(note)  ==26679==    by 0x40DC0D: SimpleTester::perform(unsigned long) (in /home/ralf/ARB-bilbo/ARB.gcc.483.NDEBUG/UNIT_TESTER/tests.slow/test_CONVERTALN_CONVERTALN_a)
(note)  ==26679==    by 0x40E24C: SimpleTester::perform_all() (in /home/ralf/ARB-bilbo/ARB.gcc.483.NDEBUG/UNIT_TESTER/tests.slow/test_CONVERTALN_CONVERTALN_a)
(note)  ==26679==    by 0x40E6FE: UnitTester::UnitTester(char const*, UnitTest_simple const*, int, unsigned long, UnitTest_simple const*) (in /home/ralf/ARB-bilbo/ARB.gcc.483.NDEBUG/UNIT_TESTER/tests.slow/test_CONVERTALN_CONVERTALN_a)
(note)  ==26679==    by 0x40CCB7: main (in /home/ralf/ARB-bilbo/ARB.gcc.483.NDEBUG/UNIT_TESTER/tests.slow/test_CONVERTALN_CONVERTALN_a)
(note) ==26679==    ORIGIN: Address 0x65c7300 is 32 bytes inside a block of size 35 alloc'd by 0x5F89C8C: __libc_start_main  (in unavailable file libc-start.c line 226)
(note)  ==26679==    by 0x40CB60: ??? (in /home/ralf/ARB-bilbo/ARB.gcc.483.NDEBUG/UNIT_TESTER/tests.slow/test_CONVERTALN_CONVERTALN_a)
(    ) ==26679==    by 0x7FEFFF647: ???
(    ) ==26679==    by 0x1B: ???
(    ) ==26679== 
[more similar following]

Note: valgrind does not report any heap problems!

comment:5 Changed 10 years ago by westram

Sanitizer reports: 

==28563==ERROR: AddressSanitizer: attempting double-free on 0x60200000c670 in thread T0:
    #0 0x2b807f250417 in __interceptor_free ../../../../gcc-4.9.0/libsanitizer/asan/asan_malloc_linux.cc:62
INCLUDE/dupstr.h:47: #1 0x452c9b in freeset<char>
INCLUDE/dupstr.h:55: #2 0x452c9b in freedup
CONVERTALN/genbank.h:30: #3 0x452c9b in GenbankRef
CONVERTALN/genbank.h:39: #4 0x452c9b in GenbankRef::operator=(GenbankRef const&)
CONVERTALN/refs.h:29: #5 0x452c9b in Refs<GenbankRef>::resize(int)
CONVERTALN/refs.h:57: #6 0x452c9b in RefContainer<GenbankRef>::resize_refs(int)
CONVERTALN/genbank.cxx:105: #7 0x452c9b in genbank_reference
CONVERTALN/genbank.cxx:289: #8 0x452c9b in GenbankParser::parse_keyed_section(char const*)
CONVERTALN/genbank.cxx:242: #9 0x455ffa in GenbankParser::parse_section()
CONVERTALN/parser.h:36: #10 0x455ffa in Parser::parse_entry()
CONVERTALN/genbank.cxx:461: #11 0x455ffa in GenbankReader::read_one_entry(Seq&)
CONVERTALN/convert.cxx:181: #12 0x4190bf in convert(FormattedFile const&, FormattedFile const&)
CONVERTALN/fconv.cxx:189: #13 0x4401d8 in test_convert
CONVERTALN/fconv.cxx:202: #14 0x440f23 in test_convert_by_format_num
CONVERTALN/fconv.cxx:297: #15 0x447d39 in TEST_SLOW_converter()
UNIT_TESTER/UnitTester.cxx:251: #16 0x40d713 in execute_guarded_ClientCode
UNIT_TESTER/UnitTester.cxx:397: #17 0x40ec14 in execute_guarded(void (*)(), long*, long, bool)
UNIT_TESTER/UnitTester.cxx:523: #18 0x40fae5 in SimpleTester::perform(unsigned long)
UNIT_TESTER/UnitTester.cxx:452: #19 0x410a70 in SimpleTester::perform_all()
UNIT_TESTER/UnitTester.cxx:629: #20 0x41129b in UnitTester::UnitTester(char const*, UnitTest_simple const*, int, unsigned long, UnitTest_simple const*)
tests.slow/test_CONVERTALN_CONVERTALN_a.cxx:24: #21 0x40ce23 in main
    #0 0x2b807f250417 in __interceptor_free ../../../../gcc-4.9.0/libsanitizer/asan/asan_malloc_linux.cc:62
INCLUDE/dupstr.h:41: #1 0x452aca in freenull<char>
CONVERTALN/genbank.h:37: #2 0x452aca in ~GenbankRef
CONVERTALN/genbank.h:39: #3 0x452aca in GenbankRef::operator=(GenbankRef const&)
CONVERTALN/refs.h:29: #4 0x452aca in Refs<GenbankRef>::resize(int)
CONVERTALN/refs.h:57: #5 0x452aca in RefContainer<GenbankRef>::resize_refs(int)
CONVERTALN/genbank.cxx:105: #6 0x452aca in genbank_reference
CONVERTALN/genbank.cxx:289: #7 0x452aca in GenbankParser::parse_keyed_section(char const*)
CONVERTALN/genbank.cxx:242: #8 0x455ffa in GenbankParser::parse_section()
CONVERTALN/parser.h:36: #9 0x455ffa in Parser::parse_entry()
CONVERTALN/genbank.cxx:461: #10 0x455ffa in GenbankReader::read_one_entry(Seq&)
CONVERTALN/convert.cxx:181: #11 0x4190bf in convert(FormattedFile const&, FormattedFile const&)
CONVERTALN/fconv.cxx:189: #12 0x4401d8 in test_convert
CONVERTALN/fconv.cxx:202: #13 0x440f23 in test_convert_by_format_num
CONVERTALN/fconv.cxx:297: #14 0x447d39 in TEST_SLOW_converter()
UNIT_TESTER/UnitTester.cxx:251: #15 0x40d713 in execute_guarded_ClientCode
UNIT_TESTER/UnitTester.cxx:397: #16 0x40ec14 in execute_guarded(void (*)(), long*, long, bool)
UNIT_TESTER/UnitTester.cxx:523: #17 0x40fae5 in SimpleTester::perform(unsigned long)
UNIT_TESTER/UnitTester.cxx:452: #18 0x410a70 in SimpleTester::perform_all()
UNIT_TESTER/UnitTester.cxx:629: #19 0x41129b in UnitTester::UnitTester(char const*, UnitTest_simple const*, int, unsigned long, UnitTest_simple const*)
tests.slow/test_CONVERTALN_CONVERTALN_a.cxx:24: #20 0x40ce23 in main
    #0 0x2b807f25062f in __interceptor_malloc ../../../../gcc-4.9.0/libsanitizer/asan/asan_malloc_linux.cc:72
CONVERTALN/global.h:127: #1 0x452836 in no_content
CONVERTALN/genbank.h:23: #2 0x452836 in GenbankRef
CONVERTALN/refs.h:27: #3 0x452836 in Refs<GenbankRef>::resize(int)
CONVERTALN/refs.h:57: #4 0x452836 in RefContainer<GenbankRef>::resize_refs(int)
CONVERTALN/genbank.cxx:105: #5 0x452836 in genbank_reference
CONVERTALN/genbank.cxx:289: #6 0x452836 in GenbankParser::parse_keyed_section(char const*)
CONVERTALN/genbank.cxx:242: #7 0x455ffa in GenbankParser::parse_section()
CONVERTALN/parser.h:36: #8 0x455ffa in Parser::parse_entry()
CONVERTALN/genbank.cxx:461: #9 0x455ffa in GenbankReader::read_one_entry(Seq&)
CONVERTALN/convert.cxx:181: #10 0x4190bf in convert(FormattedFile const&, FormattedFile const&)
CONVERTALN/fconv.cxx:189: #11 0x4401d8 in test_convert
CONVERTALN/fconv.cxx:202: #12 0x440f23 in test_convert_by_format_num
CONVERTALN/fconv.cxx:297: #13 0x447d39 in TEST_SLOW_converter()
UNIT_TESTER/UnitTester.cxx:251: #14 0x40d713 in execute_guarded_ClientCode
UNIT_TESTER/UnitTester.cxx:397: #15 0x40ec14 in execute_guarded(void (*)(), long*, long, bool)
UNIT_TESTER/UnitTester.cxx:523: #16 0x40fae5 in SimpleTester::perform(unsigned long)
UNIT_TESTER/UnitTester.cxx:452: #17 0x410a70 in SimpleTester::perform_all()
UNIT_TESTER/UnitTester.cxx:629: #18 0x41129b in UnitTester::UnitTester(char const*, UnitTest_simple const*, int, unsigned long, UnitTest_simple const*)
tests.slow/test_CONVERTALN_CONVERTALN_a.cxx:24: #19 0x40ce23 in main

comment:6 Changed 10 years ago by westram

  • Resolution set to fixed
  • Status changed from _started to closed
  • Summary changed from gcc creates invalid binaries from arb code to unittests in CONVERTALN crash when code gets optimized

by [13001]

comment:7 Changed 9 years ago by westram

  • Milestone set to arb6.1

mark changes that got fixed after arb 6.0.x

comment:8 Changed 3 years ago by westram

  • Milestone changed from arb6.1 to arb7.0

Milestone renamed

Note: See TracTickets for help on using tickets.