Skip to content

arsig: fix macOS (Apple clang) build of xgb_model.c#63

Draft
ProfessionalSeaweedDevourer wants to merge 1 commit into
PrideLab:masterfrom
ProfessionalSeaweedDevourer:fix/macos-clang-arsig-build
Draft

arsig: fix macOS (Apple clang) build of xgb_model.c#63
ProfessionalSeaweedDevourer wants to merge 1 commit into
PrideLab:masterfrom
ProfessionalSeaweedDevourer:fix/macos-clang-arsig-build

Conversation

@ProfessionalSeaweedDevourer

Copy link
Copy Markdown

Problem

On macOS, the C compiler invoked as gcc is Apple clang. Building from a
clean tree fails in arsig while compiling the generated src/arsig/xgb_model.c
(the SVM/XGBoost integer-ambiguity-validation model):

xgb_model.c:102084:298: fatal error: bracket nesting level exceeded maximum of 256
xgb_model.c:102084:298: note: use -fbracket-depth=N to increase maximum nesting level
1 error generated.

Apple clang caps C bracket nesting at 256; that generated file exceeds it.
GNU gcc (Linux) has no such default limit, which is why CI/Linux builds are
unaffected.

Why the obvious one-liner is not enough

Adding -fbracket-depth=1024 lifts the 256 limit, but clang parses via
recursive descent, and on the default macOS stack of 8 MB (ulimit -s
= 8192) the parser then overflows the stack and segfaults on this very
deeply nested file:

clang: error: unable to execute command: Segmentation fault: 11
clang: error: clang frontend command failed due to signal

So a robust fix must (a) raise the bracket-depth limit and (b) give the
compiler enough stack.

Fix (src/arsig/Makefile, no effect on GNU/Linux)

  • add -fbracket-depth=1024 only when the C compiler is clang (GNU gcc has
    no such limit and rejects the flag with unrecognized command-line option);
  • raise the stack soft limit to the hard limit for the C compile step so
    clang's recursive parser does not overflow.
CFLAGS = -std=c99
ifneq (,$(findstring clang,$(shell gcc --version 2>/dev/null)))
  CFLAGS += -fbracket-depth=1024
endif
...
$(OBJ_PATH)%.o : %.c
	$(DIR_GUARD)
	ulimit -s $$(ulimit -Hs) 2>/dev/null || true; gcc $(CFLAGS) -c $< -o $@

Testing (macOS arm64, Apple clang 17; Homebrew GNU gcc-16)

compiler flag stack result
Apple clang none (current) 8 MB fail: bracket nesting > 256
GNU gcc-16 none 8 MB OK (no limit; flag not needed)
GNU gcc-16 -fbracket-depth 8 MB fail: unrecognized option (flag must be clang-only)
Apple clang -fbracket-depth 8 MB segfault (parser stack overflow)
Apple clang -fbracket-depth hard (64 MB) OK

With this patch, a full install.sh build completes on macOS under the default
8 MB stack, and GNU gcc builds the file unchanged (flag not added).

Alternative

The stack bump could instead live once in install.sh (global ulimit -s
before the build) rather than in the arsig recipe; happy to move it there if
preferred. Opened as a draft for maintainer feedback on the approach.

On macOS the C compiler invoked as 'gcc' is Apple clang, which (1) caps C
bracket nesting at 256 and (2) parses via recursive descent. The generated
src/arsig/xgb_model.c (SVM/XGBoost ambiguity-validation model) exceeds depth
256, so the stock build fails:

    xgb_model.c: fatal error: bracket nesting level exceeded maximum of 256

Adding -fbracket-depth raises the limit, but on the default macOS 8 MB stack
clang's parser then overflows the stack and segfaults on this file. The build
only succeeds when the stack is also enlarged.

Fix (arsig/Makefile, no effect on GNU/Linux builds):
- add -fbracket-depth=1024 only when the C compiler is clang (GNU gcc has no
  such limit and rejects the flag);
- raise the stack soft limit to the hard limit for the C compile so clang's
  recursive parser does not overflow.

Verified on macOS arm64 (Apple clang 17): full install.sh build now completes
under the default 8 MB stack. GNU gcc-16 builds the file unchanged (flag not
added).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant