Building Guide (liblfds)

From liblfds.org
Revision as of 01:21, 30 December 2015 by Admin (talk | contribs) (→‎Other Toolchains)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search

Introducton

Building liblfds requires build files (makefile, Visual Studio solution files, etc) for your toolchain and a port of the abstraction layer to your platform, where a platform is a combination of compiler, operating system and processor.

This guide only deals with build files, i.e. actually getting a build building, even if it fails to build fully due to a missing porting abstraction layer. To read about the porting abstraction layer, please see the Porting Guide (liblfds).

A number of toolchains are supported out-of-the-box, and all supported toolchains also have a porting abtraction layer provided.

Supported Toolchains

The supported toolchains are;

  • GCC and gnumake (everyone and their dog's toolchain)
  • GCC, kbuild and gnumake (Linux kernel toolchain)
  • SDK for Windows 7 and gnumake (Windows user-mode toolchain - and, yes, gnumake, not nmake)
  • Visual Studio Professional 2012 (Windows user-mode toolchain)
  • Visual Studio Professional Edition 2012 and WDK 8.0 (Windows Vista and later kernel toolchain)
  • Visual Studio Professional Edition 2013 and WDK 8.1 (Windows 7 and later kernel toolchain)
  • WDK 7.1 (Windows XP kernel toolchain)

Other Toolchains

The Visual Studio solution files for Visual Studio 2012 should hopefully work fine with later versions, being automatically upgraded.

Finally, note that SDK for Windows 7 is obsolete. Microsoft no longer distribute a command-line only package of their compilers.

Directory Structure

An overview of the full directory structure, to give context, although here only the liblfds700/build/ irectory is of particular relevance;

└───liblfds700                                                   : liblfds library
    ├───bin                                                      : output directory - the library file ends up here
    ├───build                                                    : build configuration directory - contains one directory per platform
    │   ├───gcc_and_gnumake                                      : GCC and gnumake
    │   ├───gcc_kbuild_and_gnumae                                : GCC, kbuild and gnumake
    │   ├───sdk_for_windows_7_and_gnumake                        : SDK 7 and gnumake
    │   ├───visual_studio_professional_2012                      : Visual Studio Professional 2012
    │   ├───visual_studio_professional_2012_and_wdk_8.0          : Visual Studio Professional Edition 2012 and WDK 8.0
    │   ├───visual_studio_professional_2013_and_wdk_8.1          : Visual Studio Professional Edition 2013 and WDK 8.1
    │   └───wdk_7.1                                              : WDK 7.1
    ├───inc                                                      : the public header files
    │   └───liblfds700
    ├───obj                                                      : temporary directory for object files
    └───src                                                      : the data structures
        ├───lfds700_btree_addonly_unbalanced                     : btree (add-only, unbalanced)
        ├───lfds700_freelist                                     : freelist
        ├───lfds700_hash_addonly                                 : hash (add-only)
        ├───lfds700_liblfds                                      : misc library functions (init, cleanup, random number generation, etc)
        ├───lfds700_list_addonly_ordered_singlylinked            : singly-linked list (add-only, ordered, singly-linked)
        ├───lfds700_list_addonly_singlylinked_unordered          : singly-linked list (add-only, singly-linked, unordered)
        ├───lfds700_queue                                        : queue
        ├───lfds700_queue_bounded_singleconsumer_singleproducer  : queue (bounded, single-consumer, single-producer)
        ├───lfds700_ringbuffer                                   : ringbuffer
        └───lfds700_stack                                        : stack

Building

The library directory tree contains at its top level directory called build.

This directory contains one directory per supported toolchain, where each such directory contains the files necessary to build for that toolchain. Detailed descriptions of how to build for each toolchain are given below, with one heading per toolchain.

The resultant library file will be placed in the top level directory liblfds700/bin/, with the root filename liblfds700.*, where the suffix varies by build type (static or dynamic) and by operating system (.a, .lib, etc).

The header files are in the top level directory inc. They are arranged a single master header file, liblfds700/inc/liblfds700.h together with a subdirectory, liblfds700/inc/liblfds700/, which contains a set of header files, one per data structure, where these per-data structure header files are included by the master file.

To use the header files, include and only include the master header file, liblfds700/inc/liblfds700.h.

There is no installation support with any toolchain; Windows has no standard location for header files, and supporting Linux installs for static or dynamic libraries would entail making a separate toolchain directory, "linux_gcc_and_gnumake", to reflect the Linux dependency, just to permit the installation by the makefile of the libraries.

On all platforms with command line builds, you need to clean between changing build types (debug, release, static, dynamic, profiled, etc). On platforms using MSVC, there is no need to clean between changing build types.

Per-Toolchain Build Instructions

GCC and gnumake

└───liblfds700
    └───build
        └───gcc_and_gnumake
                Makefile

To build, install GCC and gnumake, enter the build directory and type;

make

The following targets are available;

bare_ar_cov     : bare, archive (.a), coverage
bare_ar_dbg     : bare, archive (.a), debug
bare_ar_prof    : bare, archive (.a), profiling
bare_ar_rel     : bare, archive (.a), release
bare_ar_tsan    : bare, archive (.a), thread sanitizer

bare_so_cov     : bare, shared (.so), coverage
bare_so_dbg     : bare, shared (.so), debug
bare_so_prof    : bare, shared (.so), profiling
bare_so_rel     : bare, shared (.so), release
bare_so_tsan    : bare, shared (.so), thread sanitizer

hosted_ar_cov   : hosted implementation, archive (.a), coverage
hosted_ar_dbg   : hosted implementation, archive (.a), debug
hosted_ar_prof  : hosted implementation, archive (.a), profiling
hosted_ar_rel   : hosted implementation, archive (.a), release
hosted_ar_tsan  : hosted implementation, archive (.a), thread sanitizer

hosted_so_cov   : hosted implementation, shared (.so), coverage
hosted_so_dbg   : hosted implementation, shared (.so), debug
hosted_so_prof  : hosted implementation, shared (.so), profiling
hosted_so_rel   : hosted implementation, shared (.so), release
hosted_so_tsan  : hosted implementation, shared (.so), thread sanitizer

clean           : what you'd expect

When switching from one target to another, clean must be made.

If building *_ar_tsan, libtsan must be installed. This is not necessary if building *_so_tsan.

Bare and hosted builds differ in and only in that for a bare build, GCC is passed the arguments "-ffreestanding -nodefaultlibs -nostdinc -nostdlib".

The single consequence of this is that a hosted build will end up including assert.h and so will assert all arguments and all functions, and assert check argument alignment where necessary.

GCC, kbuild and gnumake

└───liblfds700
    └───build
        └───gcc_kbuild_and_gnumae
                Kbuild
                Makefile

To build, install GCC, gnumake, kbuild (and the Linux kernel headers), enter the build directory and type;

make

The makefile is complimentory and simply issues the necessary kbuild command.

There are strictly speaking three targets;

 clean
 help
 modules

Note that kbuild clean has never worked properly, so it's useless, that modules is the default build (i.e. with no arguments) and help just prints help - so really there's only one target. As far as I can tell, kbuild is producing a release build; there are no options to produce other builds.

I do not properly understand kbuild. I've read the documentation; I find for example examples which have switches in which are simply not present in the documentation, and in general I don't see how libraries are built to fit in with larger projects. The most I can say about this build is that it compiles without warnings or errors. I am pretty sure it's not something which can be used as it is in a kernel build or driver, but anyone who knows enough to be developing such a thing should be able to take what is here (since it compiles against the kernel) and easily fit it into their work.

Any feedback on making this build proper is greatly appreciated.

SDK for Windows 7 and gnumake

└───liblfds700
    └───build
        └───sdk_for_windows_7_and_gnumake
                liblfds700.def
                makefile

To build, install Windows SDK 7 and gnumake, enter the build directory and type;

make

The following targets are available;

libdbg    : archive (.lib), debug
librel    : archive (.lib), release

dlldbg    : shared (.dll), debug
dllrel    : shared (.dll), release

clean     : what you'd expect

When switching from one target to another, clean must be made.

Visual Studio Professional 2012

└───liblfds700
    └───build
        └───visual_studio_professional_2012
                liblfds700.def
                liblfds700.sln
                liblfds700.vcxproj
                liblfds700.vcxproj.filters
                liblfds700.vcxproj.user

To build, install Visual Studio Professional 2012 and then load the liblfds700.sln file into Visual Studio.

The solution is configured with the following configurations;

  • Debug DLL
  • Release DLL
  • Debug LIB
  • Release LIB

And with the following platforms;

  • Win32 (which is to say, x86)
  • x64

All combinations of these configurations and platforms are supported. It is not necessary to clean between switching builds.

Visual Studio Professional Edition 2012 and WDK 8.0

└───liblfds700
    └───build
        └───visual_studio_professional_2012_and_wdk_8.0
                driver_entry.c
                liblfds700.def
                liblfds700.sln
                liblfds700.vcxproj
                liblfds700.vcxproj.filters
                liblfds700.vcxproj.user

To build, install Visual Studio Professional Edition 2012 and WDK 8.0 (and the WDF co-installers - WDK 8.0 will tell you about them when you install it), and then load the liblfds700.sln file into Visual Studio.

The following configurations are available;

  • Debug DLL
  • Release DLL
  • Debug LIB
  • Release LIB

And with the following platforms;

  • Win32 (which is to say, x86)
  • x64

All combinations of these configurations and platforms are supported. It is not necessary to clean between switching builds.

I have to say, I do not properly understand how to configure MSVC with WDK. It is not clear to me what configuration options must be changed to produce a kernel build library. I have made the changes which seem to me right, but where I have not yet produced a test driver with which I can link against the library, I am certain things are not quite right.

Any feedback on making this build proper is greatly appreciated.

Visual Studio Professional Edition 2013 and WDK 8.1

└───liblfds700
    └───build
        └───visual_studio_professional_2013_and_wdk_8.1
                driver_entry.c
                liblfds700.def
                liblfds700.sln
                liblfds700.vcxproj
                liblfds700.vcxproj.filters
                liblfds700.vcxproj.user

To build, install Visual Studio Professional Edition 2013 and WDK 8.1.

The following configurations are available;

  • Debug DLL
  • Release DLL
  • Debug LIB
  • Release LIB

And with the following platforms;

  • ARM
  • Win32 (which is to say, x86)
  • x64

All combinations of these configurations and platforms are supported. It is not necessary to clean between switching builds.

I have to say, I do not properly understand how to configure MSVC with WDK. It is not clear to me what configuration options must be changed to produce a kernel build library. I have made the changes which seem to me right, but where I have not yet produced a test driver with which I can link against the library, I am certain things are not quite right.

Any feedback on making this build proper is greatly appreciated.

WDK 7.1

└───liblfds700
    └───build
        └───wdk_7.1
                dirs
                driver_entry_renamed_to_avoid_compiler_warning.c
                liblfds700.def
                readme_before_win_kernel_build.txt
                runme_before_win_kernel_dynamic_lib_build.bat
                runme_before_win_kernel_static_lib_build.bat
                sources.dynamic
                sources.static

The WDK 7.1 kernel build environment is primitive and has a number of severe limitations; in particular, all source files must be in one directory and it is not possible to choose the output binary type (static or dynamic library) from the build command line; rather, a string has to be modified in a text file used by the build (!)

To deal with these limitations, it is necessary for a Windows kernel build to run a batch file prior to building. There are two batch files, one for static library builds and the other for dynamic library builds. They are idempotent; you can run them as often as you like and switch between them as often as you want. It's all fine.

Both batch files copy all the sources file into a single directory, liblfds700/build/wdk7.1/single_dir_for_windows_kernel/.

The static library batch file will then copy liblfds700/sources.static into liblfds700/build/wdk7.1/single_dir_for_windows_kernel/, which will cause a static library to be built.

The dynamic library batch file will then copy liblfds700/sources.dynamic into liblfds700/build/wdk7.1/single_dir_for_windows_kernel/, which will cause a dynamic library to be built. It will also copy driver_entry_renamed_to_avoid_compiler_warning.c to liblfds700/build/wdk7.1/single_dir_for_windows_kernel/driver_entry.c (note the renaming), since the linker requires the DriverEntry function to exist, even though it's not used.

To build, start a build command line as usual, indicating through the command line you select whether you are making a debug or release build. Then run the appropriate liblfds batch file to select a static or dynamic library. Then, finally, run build.exe in the liblfds700/build/wdk_7.1/ directory, with whatever arguments you prefer.

Note that due to limitations in the build environment, object files appear in a subdirectory in liblfds700/build/wdk7.1/single_dir_for_windows_kernel/, rather than in the usual liblfds700/obj/. The final binary output however still appears almost as usual, in liblfds700/bin/[processor]/, where processor might be i386. (The usual is just liblfds700/bin/, with no subdirectories).

Note that the sources file used to compile the library asserts the define _KERNEL_MODE. This appears to be necessary, due to the lack of a compiler-provided macro to differentiate in the code between a user-mode or kernel-mode build.

See Also