Llamacpp compile failed on Jetson Orin Nano (8GB) #10545

wsmzsl · 2024-11-27T12:05:42Z

wsmzsl
Nov 27, 2024

My nano (8g) is flashed with jetpack6.0 (cuda=12.2, gcc=11.4). When I compile the source code of llama.cpp (with cuda) on orin nano, the following error occurs. Has anyone successfully compiled on nano? Does anyone know how to solve this error?

/usr/lib/gcc/aarch64-linux-gnu/11/include/arm_neon.h(38): error: identifier "__Int8x8_t" is undefined
typedef __Int8x8_t int8x8_t;
^
/usr/lib/gcc/aarch64-linux-gnu/11/include/arm_neon.h(39): error: identifier "__Int16x4_t" is undefined
typedef __Int16x4_t int16x4_t;
^
/usr/lib/gcc/aarch64-linux-gnu/11/include/arm_neon.h(40): error: identifier "__Int32x2_t" is undefined
typedef __Int32x2_t int32x2_t;
^
/usr/lib/gcc/aarch64-linux-gnu/11/include/arm_neon.h(41): error: identifier "__Int64x1_t" is undefined
typedef __Int64x1_t int64x1_t;
^
/usr/lib/gcc/aarch64-linux-gnu/11/include/arm_neon.h(42): error: identifier "__Float16x4_t" is undefined
typedef __Float16x4_t float16x4_t;
^
/usr/lib/gcc/aarch64-linux-gnu/11/include/arm_neon.h(43): error: identifier "__Float32x2_t" is undefined
typedef __Float32x2_t float32x2_t;
^
/usr/lib/gcc/aarch64-linux-gnu/11/include/arm_neon.h(44): error: identifier "__Poly8x8_t" is undefined
typedef __Poly8x8_t poly8x8_t;
^
/usr/lib/gcc/aarch64-linux-gnu/11/include/arm_neon.h(45): error: identifier "__Poly16x4_t" is undefined
typedef __Poly16x4_t poly16x4_t;
......
Error limit reached.
100 errors detected in the compilation of "/home/nano8g-1/llama.cpp/ggml/src/ggml-cuda/ggml-cuda.cu".
Compilation terminated.
gmake[2]: *** [ggml/src/ggml-cuda/CMakeFiles/ggml-cuda.dir/build.make:314: ggml/src/ggml-cuda/CMakeFiles/ggml-cuda.dir/ggml-cuda.cu.o] Error 4
gmake[1]: *** [CMakeFiles/Makefile2:1760: ggml/src/ggml-cuda/CMakeFiles/ggml-cuda.dir/all] Error 2
gmake: *** [Makefile:146: all] Error 2

Romyull-Islam · 2024-12-22T23:23:03Z

Romyull-Islam
Dec 22, 2024

I am facing a similar problem when building llama.cpp for CPU in the jetson or nano. Lllama.cpp can not be built for Cortex a-78AE CPU.
I think a similar type of issue is occurring here.

0 replies

joseph777111 · 2024-12-22T23:56:18Z

joseph777111
Dec 22, 2024

I don't have this platform; however, it would be much more helpful if you guys would post the exact commands you used to try to compile and run llama.cpp on your respective platform(s). 🤔

Did you follow the instructions from https://github.com/ggerganov/llama.cpp/blob/master/docs/build.md#cuda

cmake -B build -DGGML_CUDA=ON
cmake --build build --config Release

6 replies

joseph777111 Dec 23, 2024

Nice! So you got it to run then? 🤔

Romyull-Islam Dec 23, 2024

Nice! So you got it to run then? 🤔

Yeah. It is working. The Cuda is not performing well; seems stuck. so using the CPU right now.
CPU is pretty fast for small model inference.

joseph777111 Dec 23, 2024

Nice! 😋

hitmachineWu Dec 23, 2024

Nice! So you got it to run then? 🤔

Yeah. It is working. The Cuda is not performing well; seems stuck. so using the CPU right now. CPU is pretty fast for small model inference.

Congratulations, but may I ask what versions of gcc and g++ you have locally compiled? I also want to compile the entire project on the Jetson Xavier board, but I encountered some compilation errors. Have you ever come across this issue on your side?

In file included from /home/agilex/junyuan/llama.cpp/ggml/src/ggml-cpu/ggml-cpu-quants.c:7:0:
/home/agilex/junyuan/llama.cpp/ggml/src/ggml-cpu/ggml-cpu-quants.c: In function ‘ggml_vec_dot_q2_K_q8_K’:
/home/agilex/junyuan/llama.cpp/ggml/src/ggml-cpu/ggml-cpu-impl.h:290:27: error: implicit declaration of function ‘vld1q_s16_x2’; did you mean ‘vld1q_s16’? [-Werror=implicit-function-declaration]
 #define ggml_vld1q_s16_x2 vld1q_s16_x2
                           ^

                                         ^~~~~~~~~~~~~~~~~
/home/agilex/junyuan/llama.cpp/ggml/src/ggml-cpu/ggml-cpu-impl.h:290:27: error: invalid initializer
 #define ggml_vld1q_s16_x2 vld1q_s16_x2
                           ^
/home/agilex/junyuan/llama.cpp/ggml/src/ggml-cpu/ggml-cpu-quants.c:4217:41: note: in expansion of macro ‘ggml_vld1q_s16_x2’
         const ggml_int16x8x2_t q8sums = ggml_vld1q_s16_x2(y[i].bsums);
                                         ^~~~~~~~~~~~~~~~~
/home/agilex/junyuan/llama.cpp/ggml/src/ggml-cpu/ggml-cpu-impl.h:291:27: error: implicit declaration of function ‘vld1q_u8_x2’; did you mean ‘vld1q_u32’? [-Werror=implicit-function-declaration]
 #define ggml_vld1q_u8_x2  vld1q_u8_x2
                           ^
             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from /home/agilex/junyuan/llama.cpp/ggml/src/ggml-cpu/ggml-cpu-quants.c:7:0:
/home/agilex/junyuan/llama.cpp/ggml/src/ggml-cpu/ggml-cpu-quants.c: In function ‘ggml_vec_dot_q3_K_q8_K’:
/home/agilex/junyuan/llama.cpp/ggml/src/ggml-cpu/ggml-cpu-impl.h:291:27: error: invalid initializer
 #define ggml_vld1q_u8_x2  vld1q_u8_x2
                           ^
/home/agilex/junyuan/llama.cpp/ggml/src/ggml-cpu/ggml-cpu-quants.c:4801:36: note: in expansion of macro ‘ggml_vld1q_u8_x2’
         ggml_uint8x16x2_t qhbits = ggml_vld1q_u8_x2(qh);
                                    ^~~~~~~~~~~~~~~~
/home/agilex/junyuan/llama.cpp/ggml/src/ggml-cpu/ggml-cpu-impl.h:291:27: error: invalid initializer
 #define ggml_vld1q_u8_x2  vld1q_u8_x2

Romyull-Islam Jan 3, 2025

Hi,
I am also using jetson nano with cotex A57 cpu. Here I am using gcc and g++ 9.4.0. Here I also had to update the txt file from llama.cpp/ggml/src/ggml-cpu/CMakeLists.txt.

In your case you are facing erron in the compiler for ARM Neon
ggml-cpu-impl.h

Try to make sure all the header file for amr-neon are included as header file or inside cmakelist.txt

wsmzsl · 2025-01-10T13:11:54Z

wsmzsl
Jan 10, 2025
Author

Thank you. I have solved it by updating the camke

…

---- Replied Message ---- | From | ***@***.***> | | Date | 12/23/2024 07:56 | | To | ***@***.***> | | Cc | ***@***.***>***@***.***> | | Subject | Re: [ggerganov/llama.cpp] Llamacpp compile failed on Jetson Orin Nano (8GB) (Discussion #10545) | I don't have this platform; however, it would be much more helpful if you guys would post the exact commands you used to try to compile and run llama.cpp on your respective platform(s). 🤔 Did you follow the instructions from https://github.com/ggerganov/llama.cpp/blob/master/docs/build.md#cuda cmake -B build -DGGML_CUDA=ON cmake --build build --config Release — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: ***@***.***>

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Llamacpp compile failed on Jetson Orin Nano (8GB) #10545

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Replies: 3 comments 6 replies

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

Select a reply

Llamacpp compile failed on Jetson Orin Nano (8GB) #10545

Replies: 3 comments · 6 replies

wsmzsl Jan 10, 2025 Author

Replies: 3 comments 6 replies

wsmzsl
Jan 10, 2025
Author