Difference between revisions of "Toolchain"

From NaplesPU Documentation
Jump to: navigation, search
 
(66 intermediate revisions by 5 users not shown)
Line 1: Line 1:
−
The nu+ toolchain is based on the LLVM project and includes a custom version of the Clang frontend and a native nu+ backend. The nu+ linker is based on MCLinker. In addition, Nu+ comes with an ELFtoHEX tool that takes in input the ELF file and builds memory images that can be loaded into memory.
+
The NaplesPU toolchain is a collection of tools required to compile a NaplesPU ''executable'' application. It is based on the [https://llvm.org LLVM] project leveraging on a custom version of the [[Frontend | Clang Frontend]] and on a ''from-scratch'' implementation of the [[Backend | Backend]]. The toolchain comes with a modified version of [https://lld.llvm.org LLD], an ''elf2hex'' tool to generate NaplesPU compatible memory images and lastly a custom version of the ''objdump'' tool for debuggingc purposes. A custom implementation of the ''libc'' libraries is provided.
  
−
== Requirements for Nu+ toolchain==
+
The toolchain supports the compilation of ''C''-based and ''OpenCL C'' kernels.
−
* cmake (we tested cmake 3.7.2 downloaded from https://cmake.org/download/)
 
−
* flex (we tested flex 2.6.0 downloaded using the advanced packaging tool, i.e. apt-get)
 
−
* bison (we tested bison 3.0.4 downloaded using the advanced packaging tool, i.e. apt-get)
 
−
* libz-dev (downloaded using the advanced packaging tool, i.e. apt-get)
 
  
−
== How to compile the the Nu+ toolchain==
+
== Building Toolchain on Ubuntu Linux environment ==
−
To build the nu+ toolchain, you must launch '''./setup_new.sh -n''' from a terminal in the compiler root folder. The script will create a new directory called ''build'' and will start building the compiler. By default the script tries to build the compiler using '''4''' threads. If you want to modify that, especially if your machine have less than 8GB of RAM, you can use the '''-t''' flag to specify the number of threads to use. Other information can be obtained by using the '''-h''' flag.
+
This section shows how to build the NaplesPU toolchain in an Ubuntu Linux environment. The following steps are still valid in any other Unix-based system despite package manager related differences.
  
−
== How to compile a kernel for the NuPlus architecture ==
+
=== Required Software ===
−
Some kernels are provided with the standard release of the NuPlus toolchain. We provide makefile to compile these kernels for NuPlus. In case you want to add a new kernel, it is suggested to copy a kernel folder and replace C/C++ files with your own source code. Then, remember to modify the makefile updating the SRCS variable with the names of the C/C++ files you want to compile. When using these makefiles, different tools are called:
+
The NaplesPU toolchain installation relies on the following dependencies:
−
* Clang to emit the LLVM IR and the object files
+
* Git
−
* MCLINKER to handle the job of linking
+
* GCC
−
* elf2hex to generate the HEX file from the ELF file
+
* CMake
−
* llvm-objdump to generate the dump from the elf file
+
* Python 2.7
−
In order to change the optimization level of the compiler or any other flag, it is possible to modify the misc/NuPlus.mk (or misc/Makefile.nuplus in [[http://www.mango-project.eu/ MANGO]]) makefile.
+
* libxml
 +
* Zlib
 +
* Bison
 +
* Flex
 +
* Libedit
 +
* Swig
 +
* ncurses libraries
 +
* Ninja
  
−
== How to compile the libraries for the NuPlus architecture ==
+
The following terminal command may be used to install all required software:
−
To compile the libraries you only have to execute '''./setup_new.sh -l''' from a terminal in the NuPlusLLVM root folder. The script will automatically build the libraries. Note that you must have compiled the NuPlus compiler first.
 
  
−
== Backend for a custom target architecture ==
+
<code> $ sudo apt install libxml2-dev git cmake gcc g++ python bison flex zlib1g-dev swig python-dev libedit-dev libncurses5-dev ninja-build </code>
−
Adding a new target architecture, i.e. nu+, to llvm requires four steps:
 
−
* [[Backend|write all backend files and place them into the lib/Target/NuPlus folder]]
 
−
* [[Frontend|modify the Clang frontend]]
 
−
* [[How to add a new backend to llvm|register and add the new backend to llvm]]
 
−
* [[Tools|modify and/or add all the external tools that are used by llvm, i.e. linker, disassembler, etc..]]
 
  
−
== Testing ==
+
=== Building Process ===
−
To test the code generation process, the [https://llvm.org/docs/TestingGuide.html LLVM testing infrastructure] has been used.
 
  
−
The script "run_tests" can be used to run the nu+ testing suite. The main purpose is to perform regression test, so that it is possible to determine if a change in the back-end has negative consequences.
+
First, you have to obtain the source code from the official repository by typing the following command:
  
−
The tool used is the "llvm-lit", a python script that parses the test files and executes the testing commands.
+
<code> $ git clone https://github.com/AlessandroCilardo/NaplesPU-toolchain <clone_directory> </code>
  
−
The '''code generation tests''' are contained in "/compiler/test/CodeGen/NuPlus" and cover the main LLVM IR operations. A code generation test file is a collection of function written in LLVM IR with commands directed to the llvm-lit and [https://llvm.org/docs/CommandGuide/FileCheck.html FileCheck] tools. These commands are written as comments. The main commands used are '''RUN''', '''CHECK''' and '''CHECK-LABEL'''.
+
The repository contains a helper script, ''setup.sh'' to make easier the installation process. To build a new implementation of the NaplesPU toolchain just type:
  
−
The '''RUN''' line tells lit how to run the test. If there are no RUN lines, lit will issue an error while running a test. The line syntax is similar to a shell’s syntax for pipelines. To examine output to figure out if the test passes the FileCheck tool must be used.
+
<code> $  ./setup.sh -n </code>
  
−
The RUN line used for all the tests is:
+
This command starts the compilation process, installing the toolchain in ''Release Mode'' in <code>/usr/local/llvm-npu</code>.
 +
If a debug version is required, add the <code> -d </code> flag to ''setup.sh''. You can also choose the number of threads for the compilation process, by using the <code> -t=<number_of_threads> </code> parameter.
  
−
<syntaxhighlight lang="c" line='line'>
+
At the end of the compilation process, libraries are required to be linked to the installation folder:
−
; RUN: llc -march=nuplus < %s | FileCheck %s
 
−
</syntaxhighlight>
 
  
−
This tells lit to run '''llc''' with nuplus architecture as target, to give the output file to the FileCheck tool.
+
<code> $  ./setup.sh -l </code>
  
−
The '''CHECK_LABEL''' and the '''CHECK''' lines are interpreted by the FileCheck tool. It compares the llc output file with the '''CHECK_LABEL''' and '''CHECK''' lines. The comparison is done in sequence, although there are directives that let the checking in not sequential order, see [https://llvm.org/docs/CommandGuide/FileCheck.html] for further informations.
+
Now, you can use the toolchain to build your own application.
  
−
== Compiler Extensions ==
+
== The NaplesPU LLVM Structure ==
−
* [[OpenCL|OpenCL extension for the nu+ architecture]]
+
The NaplesPU toolchain relies on the LLVM project, version 7.0,  providing a custom implementation of its libraries in order to make possible the generation of NaplesPU kernels.
−
* [[OpenMP|OpenMP extension for the nu+ architecture]]
 
  
 +
[http://clang.llvm.org Clang] is the compiler frontend for NaplesPU, extended to handle the token-recognition of custom intrinsic functions.
  
 +
[http://llvm.org/ LLVM] core libraries are used to implement a custom backend for NaplesPU, to manage the code lowering on the target device.
  
 +
[http://lld.llvm.org LLD] is adapted to match NaplesPU architecture requirements for linking.
  
−
[[compiler|The nu+ Compiler]]
+
[http://llvm.org/docs/CommandGuide/llvm-objdump.html objdump] is provided to disassemble and analyze the generated code.
  
−
[[libraries|The nu+ libraries]]
+
''elf2hex'' is a tool required for memory image generation.
 +
 
 +
 
 +
Check the following links for detailed informations on how the toolchain is implemented.
 +
*  [[nu+ Clang Documentation | ''NaplesPU '' Clang Documentation]]
 +
*  [[nu+ LLVM Documentation | ''NaplesPU '' LLVM Documentation]]
 +
*  [[nu+ LLD Linker| ''NaplesPU '' LLD Linker Documentation]]
 +
*  [[nu+ Tools | ''NaplesPU '' Tools]]
 +
*  [[nu+ Libraries | ''NaplesPU '' Libraries]]
 +
 
 +
== Extensions ==
 +
 
 +
* [[Extending NaplesPU for 64-bit support]]
 +
* [[Extending NaplesPU for OpenCL support]]

Latest revision as of 13:49, 29 June 2019

The NaplesPU toolchain is a collection of tools required to compile a NaplesPU executable application. It is based on the LLVM project leveraging on a custom version of the Clang Frontend and on a from-scratch implementation of the Backend. The toolchain comes with a modified version of LLD, an elf2hex tool to generate NaplesPU compatible memory images and lastly a custom version of the objdump tool for debuggingc purposes. A custom implementation of the libc libraries is provided.

The toolchain supports the compilation of C-based and OpenCL C kernels.

Building Toolchain on Ubuntu Linux environment

This section shows how to build the NaplesPU toolchain in an Ubuntu Linux environment. The following steps are still valid in any other Unix-based system despite package manager related differences.

Required Software

The NaplesPU toolchain installation relies on the following dependencies:

  • Git
  • GCC
  • CMake
  • Python 2.7
  • libxml
  • Zlib
  • Bison
  • Flex
  • Libedit
  • Swig
  • ncurses libraries
  • Ninja

The following terminal command may be used to install all required software:

$ sudo apt install libxml2-dev git cmake gcc g++ python bison flex zlib1g-dev swig python-dev libedit-dev libncurses5-dev ninja-build

Building Process

First, you have to obtain the source code from the official repository by typing the following command:

$ git clone https://github.com/AlessandroCilardo/NaplesPU-toolchain <clone_directory>

The repository contains a helper script, setup.sh to make easier the installation process. To build a new implementation of the NaplesPU toolchain just type:

$ ./setup.sh -n

This command starts the compilation process, installing the toolchain in Release Mode in /usr/local/llvm-npu. If a debug version is required, add the -d flag to setup.sh. You can also choose the number of threads for the compilation process, by using the -t=<number_of_threads> parameter.

At the end of the compilation process, libraries are required to be linked to the installation folder:

$ ./setup.sh -l

Now, you can use the toolchain to build your own application.

The NaplesPU LLVM Structure

The NaplesPU toolchain relies on the LLVM project, version 7.0, providing a custom implementation of its libraries in order to make possible the generation of NaplesPU kernels.

Clang is the compiler frontend for NaplesPU, extended to handle the token-recognition of custom intrinsic functions.

LLVM core libraries are used to implement a custom backend for NaplesPU, to manage the code lowering on the target device.

LLD is adapted to match NaplesPU architecture requirements for linking.

objdump is provided to disassemble and analyze the generated code.

elf2hex is a tool required for memory image generation.


Check the following links for detailed informations on how the toolchain is implemented.

Extensions

Retrieved from "http://www.naplespu.com/doc/index.php?title=Toolchain&oldid=2251"