



General

See also #Commands, Storage/Files, Distros, Debugging, #Tmux, GUI, etc.

Mainly Linux.





https://en.wikipedia.org/wiki/Multics - Multiplexed Information and Computing Service, is an influential early time-sharing operating system, based around the concept of a single-level memory. Virtually all modern operating systems were heavily influenced by Multics, often through Unix, which had been created by the people who had worked on Multics—either directly (Linux, macOS) or indirectly (Microsoft Windows NT).

Multics - the story of the Multics operating system for people interested in the system's history, especially Multicians. The site's goals are to preserve the technical ideas and advances of Multics so others don't need to reinvent them. record the history of Multics, its builders, and its users before we all forget. give credit where it's due for important innovations. remember some good times and good people.





List of UNIX alternatives with desirable capabilities - "People often ask if there's examples of OS's better than UNIX. This comes up when critics like me bash its design choices or especially if someone posts UNIX Hater's Handbook. Here's a few, esp from past, that give you an idea that did things differently in a way worth copying in UNIX or another OS project."[1]





Community

See also Chat#IRC - Freenode









Troubleshooting

Arch wiki: General troubleshooting

LinuxCommand.org - Now what? You have Linux installed and running. The GUI is working fine, but you are getting tired of changing your desktop themes. You keep seeing this "terminal" thing.

man pages

https://en.wikipedia.org/wiki/Man_page - short for manual page) is a form of software documentation usually found on a Unix or Unix-like operating system. Topics covered include computer programs (including library and system calls), formal standards and conventions, and even abstract concepts. A user may invoke a man page by issuing the man command. By default, man typically uses a terminal pager program such as more or less to display its output.

man man # rtfm.





Linux man pages online - links from this page display HTML renderings of the man pages from the Linux man-pages project as well as a curated collection of pages from various other free software projects. List of all man pages: by section, alphabetically, by project, by Individual sections (page names only)

Linux man-pages project - documents the Linux kernel and C library interfaces that are employed by user-space programs. With respect to the C library, the primary focus is the GNU C library (glibc), although, where known, documentation of variations on other C libraries available for Linux is also included. The project provides manual pages in the following sections:

User commands; man-pages includes a very few Section 1 pages that document programs supplied by the GNU C library. System calls documents the system calls provided by the Linux kernel. Library functions documents the functions provided by the standard C library. Devices documents details of various devices, most of which reside in /dev. Files describes various file formats, and includes proc(5), which documents the /proc file system. Overviews, conventions, and miscellaneous. Superuser and system administration commands; man-pages includes a very few Section 8 pages that document programs supplied by the GNU C library.









man --html="google-chrome-stable file%c//%s" --html cd # load the manual into chrome browser

man whoami | man2html # a Perl program that reads formatted nroff text from standard input (stdin) and writes a HTML document to standard output (stdout). A CGI program is provided to allow hypertext navigation of a system's manpages via a web browser.





https://github.com/gapan/gtkman - a simple GTK+2 manual page viewer, written in Python. Manual pages are viewed by specifying their name and optionally the section they are in, just as with the original man command. The manual pages are displayed in simple text form using the default system monospace font.





whatis ls # list the various manual pages for the ls command





apropos [string] # search the whatis database for strings

apropos ls lists all manual pages that contain ls in their title.









mangle.sh - Mangle man pages to show just the parts you need (suitable for aliasing to "man") [3]





bro pages are a highly readable supplement to man pages with concise, common-case examples for Unix commands





http://manpages.bsd.lv - creating man pages

mandoc - a suite of tools compiling mdoc, the roff macro language of choice for BSD manual pages, and man, the predominant historical language for UNIX manuals. It is small, ISO C, ISC-licensed, and quite fast. The main component of the toolset is the mandoc utility program, based on the libmandoc validating compiler, to format output for UTF-8 and ASCII UNIX terminals, HTML 5, PostScript, and PDF. mandoc has predominantly been developed on OpenBSD and is both an OpenBSD and a BSD.lv project. We strive to support all interested free operating systems, in particular FreeBSD, NetBSD, DragonFly, illumos, Minix 3, and both GNU- and musl-based Linux distributions, as well as all systems running the pkgsrc portable package build system. To support mandoc development, consider donating to the OpenBSD foundation.





Creating a man page from markdown with pandoc - How to create a man page from a README.md file









https://github.com/denisidoro/navi - An interactive cheatsheet tool for the command-line





~sircmpwn/scdoc - sourcehut git - a simple man page generator for POSIX systems written in C99.

UNIX

https://en.wikipedia.org/wiki/Unix - a family of multitasking, multiuser computer operating systems that derive from the original AT&T Unix, development starting in the 1970s at the Bell Labs research center by Ken Thompson, Dennis Ritchie, and others.

Initially intended for use inside the Bell System, AT&T licensed Unix to outside parties in the late 1970s, leading to a variety of both academic and commercial Unix variants from vendors like the University of California, Berkeley (BSD), Microsoft (Xenix), IBM (AIX), and Sun Microsystems (Solaris). In the early 1990s, AT&T sold its rights in Unix to Novell, which then sold its Unix business to the Santa Cruz Operation (SCO) in 1995.[4] The UNIX trademark passed to The Open Group, a neutral industry consortium, which allows the use of the mark for certified operating systems that comply with the Single UNIX Specification (SUS). As of 2014, the Unix version with the largest installed base is Apple's macOS.

Unix systems are characterized by a modular design that is sometimes called the "Unix philosophy". This concept entails that the operating system provides a set of simple tools that each perform a limited, well-defined function, with a unified filesystem as the main means of communication, and a shell scripting and command language to combine the tools to perform complex workflows. Unix distinguishes itself from its predecessors as the first portable operating system: almost the entire operating system is written in the C programming language, thus allowing Unix to reach numerous platforms.





https://en.wikipedia.org/wiki/*nix - or Unix-like operating system is one that behaves in a manner similar to a Unix system, while not necessarily conforming to or being certified to any version of the Single UNIX Specification. A Unix-like application is one that behaves like the corresponding Unix command or shell. There is no standard for defining the term, and some difference of opinion is possible as to the degree to which a given operating system or application is "Unix-like".

Guides

















History

See also Distros#BSD

The Creation of the UNIX* Operating System - saved from (now missing on) bell-labs.com













Research UNIX

https://en.wikipedia.org/wiki/Research_Unix - a term used to refer to versions of the Unix operating system for DEC PDP-7, PDP-11, VAX and Interdata 7/32 and 8/32 computers, developed in the Bell Labs Computing Science Research Center (frequently referred to as Department 1127).

The term Research Unix first appeared in the Bell System Technical Journal (Vol. 57, No. 6, Pt. 2 Jul/Aug 1978) to distinguish it from other versions internal to Bell Labs (such as PWB/UNIX and MERT) whose code-base had diverged from the primary CSRC version. However, that term was little-used until Version 8 Unix, but has been retroactively applied to earlier versions as well. Prior to V8, the operating system was most commonly called simply UNIX (in caps) or the UNIX Time-Sharing System.

AT&T licensed Version 5 to educational institutions, and Version 6 also to commercial sites. Schools paid $200 and others $20,000, discouraging most commercial use, but Version 6 was the most widely used version into the 1980s. Research Unix versions are often referred to by the edition of the manual that describes them, because early versions and the last few were never officially released outside of Bell Labs, and grew organically. So, the first Research Unix would be the First Edition, and the last the Tenth Edition. Another common way of referring to them is Version x (or Vx) Unix, where x is the manual edition. All modern editions of Unix—excepting Unix-like implementations such as Coherent, Minix, and Linux—derive from the 7th Edition.

Starting with the 8th Edition, versions of Research Unix had a close relationship to BSD. This began by using 4.1cBSD as the basis for the 8th Edition. In a Usenet post from 2000, Dennis Ritchie described these later versions of Research Unix as being closer to BSD than they were to UNIX System V, which also included some BSD code.





https://en.wikipedia.org/wiki/Multi-Environment_Real-Time - was a hybrid time-sharing/real-time operating system developed in the 1970s at Bell Labs for use in embedded minicomputers (in particular PDP-11s). It was later renamed UNIX Real-Time (UNIX-RT). A version called Duplex Multi Environment Real Time (DMERT) was the operating system for the AT&T 3B20D telephone switching minicomputer, designed for high availability; DMERT was later renamed Unix RTR (Real-Time Reliable).









https://en.wikipedia.org/wiki/Version_6_Unix - also called Version 6 Unix or just V6, was the first version of the Unix operating system to see wide release outside Bell Labs. It was released in May 1975 and, like its direct predecessor, targeted the DEC PDP-11 family of minicomputers. It was superseded by Version 7 Unix in 1978/1979, although V6 systems remained in regular operation until at least 1985.





https://en.wikipedia.org/wiki/xv6 - xv6 is a modern reimplementation of Sixth Edition Unix in ANSI C for multiprocessor x86 systems. It is used for pedagogical purposes in MIT's Operating Systems Engineering (6.828) course.





https://en.wikipedia.org/wiki/PWB/UNIX - an early, now discontinued, version of the Unix operating system created in the Bell Labs Computer Science Research Group of AT&T. Its stated goal was to provide a time-sharing working environment for large groups of programmers, writing software for larger batch processing computers. Two major releases of Programmer's Workbench were produced. PWB/UNIX 1.0, released July 1, 1977 was based on Version 6 Unix; PWB 2.0 was based on Version 7 Unix. The operating system was advertised by Bell System Software as late as 1981 and edition 1.0 was still on an AT&T price list for educational institutions in 1984. Most of PWB/UNIX was later incorporated in the commercial UNIX System III and UNIX System V releases.





https://en.wikipedia.org/wiki/Version_7_Unix - an important early release of the Unix operating system. V7, released in 1979, was the last Bell Laboratories release to see widespread distribution before the commercialization of Unix by AT&T Corporation in the early 1980s. V7 was originally developed for Digital Equipment Corporation's PDP-11 minicomputers and was later ported to other platforms.

V7/x86 - x86 port of UNIX V7

The Evolution of the Unix Time-sharing System - D. M. Ritchie and K. Thompson, 1979

https://en.wikipedia.org/wiki/UNIX/32V - an early version of the Unix operating system from Bell Laboratories, released in June 1979. 32V was a direct port of the Seventh Edition Unix to the DEC VAX architecture.





https://en.wikipedia.org/wiki/Ancient_UNIX - a term coined by Santa Cruz Operation,[citation needed] to describe early releases of the Unix code base released prior to Unix System III, particularly the Research Unix releases prior to and including Version 7 (the base for UNIX/32V as well as later developments of AT&T Unix).

System III / V

https://en.wikipedia.org/wiki/UNIX_System_III - discontinued version of the Unix operating system released by AT&T's Unix Support Group (USG).

AT&T announced System III in late 1981, and it was first released outside of Bell Labs in 1982. UNIX System III was a mix of various AT&T Unixes: PWB/UNIX 2.0, CB UNIX 3.0, UNIX/TS 3.0.1 and UNIX/32V. System III supported the DEC PDP-11 and VAX computers.

The system was apparently called System III because it was considered the outside release of UNIX/TS 3.0.1 and CB UNIX 3 which were internally supported Bell Labs Unices; its manual refers to it as UNIX Release 3.0 and there were no Unix versions called System I or System II. There was no official release of UNIX/TS 4.0 (which would have been System IV) either, so System III was succeeded by System V, based on UNIX/TS 5.0.





https://en.wikipedia.org/wiki/UNIX_System_V - pronounced: "System Five", is one of the first commercial versions of the Unix operating system. It was originally developed by AT&T and first released in 1983. Four major versions of System V were released, numbered 1, 2, 3, and 4. System V Release 4, or SVR4, was commercially the most successful version, being the result of an effort, marketed as "Unix System Unification", which solicited the collaboration of the major Unix vendors. It was the source of several common commercial Unix features. System V is sometimes abbreviated to SysV.





BSD

https://en.wikipedia.org/wiki/Berkeley_Software_Distribution - a Unix operating system derivative developed and distributed by the Computer Systems Research Group (CSRG) of the University of California, Berkeley, from 1977 to 1995. Today the term "BSD" is often used non-specifically to refer to any of the BSD descendants which together form a branch of the family of Unix-like operating systems. Operating systems derived from the original BSD code remain actively developed and widely used.

In the 1980s, BSD was widely adopted by vendors of workstation-class systems in the form of proprietary Unix variants such as DEC ULTRIX and Sun Microsystems SunOS. Although these proprietary BSD derivatives were largely superseded by the UNIX System V Release 4 and OSF/1 systems in the 1990s (both of which incorporated BSD code and are the basis of other modern Unix systems), later BSD releases provided a basis for several open source development projects that are ongoing, e.g., FreeBSD, OpenBSD, NetBSD, Darwin, and TrueOS. These, in turn, have been incorporated in whole or in part in modern proprietary operating systems, e.g., the TCP/IP networking code in Windows NT 3.1 and most of the foundation of Apple's macOS and iOS.

to sort

Unix at 25 - 1994





unix-jun72 - Restoration of 1st Edition UNIX kernel sources from pdf document.









http://minnie.tuhs.org/pipermail/tuhs/2017-March/009354.html [TUHS] Alcatel-Lucent releases source for 8th, 9th and 10th editions of Unix [20] [21] - v8, v9, v10







Ulix - Literate Unix, is a Unix-like operating system that was developed at University of Erlangen-Nuremberg. We used Donald E. Knuth's concept of Literate Programming for the implementation and documentation. The intention was to create a fully working system which can be used in operating system courses to show students how OS concepts (such as paging and scheduling) can be implemented. Literate programs are very accessible because they can be read like a book; the order of presentation is not enforced by program logic or compiler restrictions, but instead is guided by the implementer's creative process. Ulix was written in C and Assembler for the Intel x86 architecture; for literate programming we used Norman Ramsey's noweb tool.





POSIX / SUS

POSIX, an acronym for "Portable Operating System Interface", is a family of standards specified by the IEEE for maintaining compatibility between operating systems. POSIX defines the application programming interface (API), along with command line shells and utility interfaces, for software compatibility with variants of Unix and other operating systems. http://www.tedunangst.com/flak/post/strict-posix-nonconformance







UniForum - In 1981, the Unix platform was just emerging and there were no centralized resources in place to support this growth environment. UniForum was founded by a group of engineers and entrepreneurs who sought to provide a forum for individuals and companies interested in Unix and open systems. UniForum was born in 1980 as /usr/group, with the distinct mission to make Unix a standardized operating system that industry would adopt and use, within an open competitive framework. Later, /usr/group evolved into UniForum as a reflection of what we had become: a thriving community of open systems professionals. In March 1997, UniForum merged its operations and staff with The Open Group.





http://en.wikipedia.org/wiki/Single_UNIX_Specification - the collective name of a family of standards for computer operating systems, compliance with which is required to qualify for to use the "UNIX" trademark. The core specifications of the SUS are developed and maintained by the Austin Group, which is a joint working group of IEEE, ISO JTC 1 SC22 and The Open Group.





UNIX® Standards - The Open Group

IEEE SA - POSIX https://en.wikipedia.org/wiki/Austin_Group - or the Austin Common Standards Revision Group is a joint technical working group formed to develop and maintain a common revision of POSIX.1 and parts of the Single UNIX Specification.







https://github.com/geoff-nixon/posix-unix-standard - You can indeed get a legal, free full copy of the standard, without being extorted by an international standards body.





POSIX.1-2008 (IEEE Std 1003.1™-2008 / The Open Group Technical Standard Base Specifications, Issue 7). The 2013 edition incorporates Technical Corrigendum 1 addressing problems discovered since the approval of the 2008 edition.

POSIX.1-2008 defines a standard operating system interface and environment, including a command interpreter (or “shell”), and common utility programs to support applications portability at the source code level. POSIX.1-2008 is intended to be used by both application developers and system implementors and comprises four major components (each in an associated volume):

General terms, concepts, and interfaces common to all volumes of this standard, including utility conventions and C-language header definitions, are included in the Base Definitions volume.

Definitions for system service functions and subroutines, language-specific system services for the C programming language, function issues, including portability, error handling, and error recovery, are included in the System Interfaces volume.

Definitions for a standard source code-level interface to command interpretation services (a “shell”) and common utility programs for application programs are included in the Shell and Utilities volume.

Extended rationale that did not fit well into the rest of the document structure, which contains historical information concerning the contents of POSIX.1-2008 and why features were included or discarded by the standard developers, is included in the Rationale (Informative) volume.

The following areas are outside the scope of POSIX.1-2008:

Graphics interfaces

Database management system interfaces

Record I/O considerations

Object or binary code portability

System configuration and resource availability





Open POSIX Test Suite - a test suite for POSIX 2001 APIs, not tied to specific implementations. It provides conformance, functional, and stress testing. Initial focus is on Threads, Clocks & Timers, Signals, Message Queues, and Semaphores.













MINIX

https://en.wikipedia.org/wiki/MINIX - a POSIX-compliant (since version 2.0), Unix-like computer operating system based on a microkernel architecture. MINIX was first released in 1987, with its complete source code made available to universities for study in courses and research. It has been free and open-source software since it was re-licensed under the BSD license in April 2000.[

Linux

Linux/README - These are the release notes for Linux version 4. Read them carefully, as they tell you what this is all about, explain how to install the kernel, and what to do if something goes wrong.

Linux is a clone of the operating system Unix, written from scratch by Linus Torvalds with assistance from a loosely-knit team of hackers across the Net. It aims towards POSIX and Single UNIX Specification compliance. It has all the features you would expect in a modern fully-fledged Unix, including true multitasking, virtual memory, shared libraries, demand loading, shared copy-on-write executables, proper memory management, and multistack networking including IPv4 and IPv6. It is distributed under the GNU General Public License - see the accompanying COPYING file for more details.













News

LWN.net - a reader-supported news site dedicated to producing the best coverage from within the Linux and free software development communities.

Guides









Read Just Enough Linux - aims at getting you comfortable with some of the commands used in Linux and to hopefully help you 'get' Linux.

The Linux Command Line - A Book By William E. Shotts, Jr.





CliRef.md - linux guide





https://github.com/trimstray/the-book-of-secret-knowledge - A collection of inspiring lists, manuals, cheatsheets, blogs, hacks, one-liners, cli/web tools and more.





eLinux wiki - to preserve and present information about the development and use of Linux in embedded systems as well as open source projects and tools for general embedded development

History

Kernel

The Linux Kernel Archives - Linux Kernel Organization is a California Public Benefit Corporation established in 2002 to distribute the Linux kernel and other Open Source software to the public without charge. The Linux Kernel Organization is managed by The Linux Foundation, which provides full technical, financial and staffing support for running and maintaining the kernel.org infrastructure.









IBM: Anatomy of the Linux kernel - History and architectural decomposition

The Linux kernel - Andries Brouwer





linux-insides - A series of posts about the linux kernel and its insides.





Gittup - This is an entire(-ish) linux distribution in git. Everything is built with tup. That's why it's called gittup. You get the whole development history (where applicable) as part of the standard distribution.

















The Cathedral and the Bazaar - Eric S. Raymond anatomizes a successful open-source project, fetchmail, that was run as a deliberate test of the surprising theories about software engineering suggested by the history of Linux. I discuss these theories in terms of two fundamentally different development styles, the ``cathedral model of most of the commercial world versus the ``bazaar model of the Linux world. I show that these models derive from opposing assumptions about the nature of the software-debugging task. I then make a sustained argument from the Linux experience for the proposition that ``Given enough eyeballs, all bugs are shallow, suggest productive analogies with other self-correcting systems of selfish agents, and conclude with some exploration of the implications of this insight for the future of software. [32]













https://en.wikipedia.org/wiki/System.map - a symbol table used by the kernel. A symbol table is a look-up between symbol names and their addresses in memory. A symbol name may be the name of a variable or the name of a function. The System.map is required when the address of a symbol name, or the symbol name of an address, is needed. It is especially useful for debugging kernel panics and kernel oopses. The kernel does the address-to-name translation itself when CONFIG_KALLSYMS is enabled so that tools like ksymoops are not required.









Building













https://en.wikipedia.org/wiki/menuconfig - one of five similar tools that can configure Linux source, a necessary early step needed to compile the source code. make menuconfig, with a menu-driven user interface, allows the user to choose the features of Linux (and other options) that will be compiled. It is normally invoked using the command make menuconfig, menuconfig is a target in Linux Makefile. See also config, nconfig, gconfig, xconfig.





https://wiki.archlinux.org/index.php/Modprobed-db - keeps a running list of ALL modules ever probed on a system and allow for easy recall. This is very useful for users wishing to build a minimal kernel via a make localmodconfig which simply takes every module currently probed and switches everything BUT them off in the .config for a kernel resulting in smaller kernel packages and reduced compilation times.

Linux Kernel Seeds - pre-made kernel seeds to help you get your Linux system up and running.





https://github.com/distcc/distcc - a program to distribute builds of C, C++, Objective C or Objective C++ code across several machines on a network to speed up building. It should always generate the same results as a local build, is simple to install and use, and is usually much faster than a local compile. Further, one can use it together with native Arch build tools such as makepkg. https://wiki.archlinux.org/index.php/Distcc



Modules

https://en.wikipedia.org/wiki/Loadable_kernel_module - or LKM, an object file that contains code to extend the running kernel, or so-called base kernel, of an operating system. LKMs are typically used to add support for new hardware (as device drivers) and/or filesystems, or for adding system calls. When the functionality provided by a LKM is no longer required, it can be unloaded in order to free memory and other resources.

Kernel modules are pieces of code that can be loaded and unloaded into the kernel upon demand. They extend the functionality of the kernel without the need to reboot the system.





https://en.wikipedia.org/wiki/lsmod - a command on Linux systems which prints the contents of the /proc/modules file. It shows which loadable kernel modules are currently loaded. http://linux.die.net/man/8/lsmod



modprobe is a Linux program originally written by Rusty Russell and used to add a loadable kernel module (LKM) to the Linux kernel or to remove an LKM from the kernel. It is commonly used indirectly: udev relies upon modprobe to load drivers for automatically detected hardware. http://linux.die.net/man/8/modprobe



http://linux.die.net/man/8/insmod - trivial program to insert a module into the kernel: if the filename is a hyphen, the module is taken from standard input. Most users will want to use modprobe(8) instead, which is more clever.

http://linux.die.net/man/8/rmmod - a trivial program to remove a module from the kernel. Most users will want to use modprobe(8) with the -r option instead.





http://linux.die.net/man/8/modinfo - program to show information about a Linux Kernel module

depmod - handle dependency descriptions for loadable kernel modules

Dynamic Kernel Module Support (DKMS) is a framework used to generate Linux kernel modules whose sources do not generally reside in the Linux kernel source tree. DKMS enables kernel device drivers to be automatically rebuilt when a new kernel is installed. An essential feature of DKMS is that it automatically recompiles all DKMS modules if a new kernel version is installed. This allows drivers and devices outside of the mainline kernel to continue working after a Linux kernel upgrade. Another benefit of DKMS is that it allows the installation of a new driver on an existing system, running an arbitrary kernel version, without any need for manual compilation or precompiled packages provided by the vendor. http://linux.dell.com/dkms/manpage.html







Granary - a framework for creating flexible and efficient tools that analyse or debug arbitrary, binary Linux kernel modules. Granary uses dynamic binary translation (DBT) to dynamically rewrite and comprehensively instrument kernel modules. Granary makes it easy to create flexible tools by supporting context-aware runtime code specialisation, and by integrating high-level static analysis information with low-level instruction manipulation. Module analysis tools built on Granary are efficient because of Granary’s adoption of a relaxed transparency model, and because of Granary’s ability to instrument module code without imposing overhead on non-module kernel code. https://github.com/Granary/granary2



https://wiki.archlinux.org/index.php/modprobed-db - keeps a running list of ALL modules ever probed on a system and allow for easy recall. This is very useful for users wishing to build a minimal kernel via a make localmodconfig which simply takes every module currently probed and switches everything BUT them off in the .config for a kernel resulting in smaller kernel packages and reduced compilation times.

Drivers

http://www.linux-drivers.org/ Linux Hardware Compatibility Lists & Linux Drivers

LKDDb is an attempt to build a comprensive database of hardware and protocols know by Linux kernels. The driver database includes numeric identifiers of hardware, the kernel configuration menu needed to build the driver and the driver filename. The database is build automagically from kernel sources, so it is very easy to have always the database updated.





Firmware for PCs falls into four categories:

Updates to the CPU to work around errata, usually referred to as microcode.

Firmware for video controllers. On x86 machines this seems to only apply to ATI devices : Radeons require firmware to be able to use KMS (kernel modesetting - the preferred option) as well as for Xorg. For earlier radeon chips (before the R600), the firmware is still in the kernel.

Firmware updates for wired network ports. Mostly they work even without the updates, but one must assume that they will work better with the updated firmware.

Firmware for other devices, such as wifi. These devices are not required for the PC to boot, but need the firmware before these devices can be used.





http://en.wikipedia.org/wiki/Dmesg - display message or driver message, a command on most Linux and Unix-based operating systems that prints the message buffer of the kernel. http://www.freebsd.org/cgi/man.cgi?dmesg http://www.linfo.org/dmesg.html dmesg explained http://inconsolation.wordpress.com/2013/08/11/dmesg-an-old-new-cool-tool/











Parameters

There are three ways to pass options to the kernel and thus control its behaviour:

When building the kernel.

When starting the kernel (usually, when invoked from a boot loader).

At runtime (through the files in /proc and /sys). See sysctl for details.

bootparam - introduction to boot time parameters of the Linux kernel





https://en.wikipedia.org/wiki/Sysctl - an interface for examining and dynamically changing parameters in Unix-like operating systems. The BSD implementation of sysctl uses system calls or system call wrappers, while the Linux implementation primarily uses files contained in a virtual file system.

Linux Journal: The sysctl Interface

Deprecation of /etc/sysctl.conf From version 207 on, systemd will not apply the settings from /etc/sysctl.conf anymore: it will only apply those from /etc/sysctl.d/*. Since the settings of our /etc/sysctl.conf shipped by procps-ng have become kernel defaults anyway, we have decided to deprecate this file.

cat /proc/sys/net/ipv4/ip_forward

Messages





Timers

kernel.org: hrtimers - subsystem for high-resolution kernel timers, the Linux Kernel documentation.

Sources

Kernel patchsets

linux-zen

https://liquorix.net/ - version of zen with configs

linux-rt

linux-ck

linux-pf

Nvidia 340 drivers not available.

linux-xanmod

XanMod Kernel - a general-purpose Linux kernel distribution with custom settings and new features.Built to provide a more rock-solid, responsive and smooth desktop experience.Supports all recent 64-bit versions of Debian and Ubuntu-based systems.

Hardware





Standards

Linux Standard Base

Linux Standard Base (LSB) is a joint project by several Linux distributions under the organizational structure of the Linux Foundation to standardize the software system structure, including the filesystem hierarchy, used with Linux operating system. The LSB is based on the POSIX specification, the Single UNIX Specification, and several other open standards, but extends them in certain areas. http://en.wikipedia.org/wiki/Linux_Standard_Base







freedesktop.org

freedesktop.org is open source / open discussion software projects working on interoperability and shared technology for X Window System desktops. The most famous X desktops are GNOME and KDE, but developers working on any Linux/UNIX GUI technology are welcome to participate. freedesktop.org is building a base platform for desktop software on Linux and UNIX. The elements of this platform have become the backend for higher-level application-visible APIs such as Qt, GTK+, XUL, VCL, WINE, GNOME, and KDE. The base platform is both software and specifications. http://en.wikipedia.org/wiki/Freedesktop.org



$XDG_DATA_HOME defines the base directory relative to which user specific data files should be stored. If $XDG_DATA_HOME is either not set or empty, a default equal to $HOME/.local/share should be used. $XDG_CONFIG_HOME defines the base directory relative to which user specific configuration files should be stored. If $XDG_CONFIG_HOME is either not set or empty, a default equal to $HOME/.config should be used. $XDG_DATA_DIRS defines the preference-ordered set of base directories to search for data files in addition to the $XDG_DATA_HOME base directory. The directories in $XDG_DATA_DIRS should be seperated with a colon ':'. If $XDG_DATA_DIRS is either not set or empty, a value equal to /usr/local/share/:/usr/share/ should be used. $XDG_CONFIG_DIRS defines the preference-ordered set of base directories to search for configuration files in addition to the $XDG_CONFIG_HOME base directory. The directories in $XDG_CONFIG_DIRS should be seperated with a colon ':'. If $XDG_CONFIG_DIRS is either not set or empty, a value equal to /etc/xdg should be used.

The order of base directories denotes their importance; the first directory listed is the most important. When the same information is defined in multiple places the information defined relative to the more important base directory takes precedent. The base directory defined by $XDG_DATA_HOME is considered more important than any of the base directories defined by $XDG_DATA_DIRS. The base directory defined by $XDG_CONFIG_HOME is considered more important than any of the base directories defined by $XDG_CONFIG_DIRS.





xdg-user-dirs is a tool to help manage "well known" user directories like the desktop folder and the music folder. It also handles localization (i.e. translation) of the filenames. https://wiki.archlinux.org/index.php/Xdg_user_directories















/etc/machine-id - contains the unique machine ID of the local system that is set during installation or boot. The machine ID is a single newline-terminated, hexadecimal, 32-character, lowercase ID. When decoded from hexadecimal, this corresponds to a 16-byte/128-bit value. This ID may not be all zeros.The machine ID is usually generated from a random source during system installation or first boot and stays constant for all subsequent boots. Optionally, for stateless systems, it is generated during runtime during early boot if necessary.The machine ID may be set, for example when network booting, with the systemd.machine_id= kernel command line parameter or by passing the option --machine-id= to systemd. An ID is specified in this manner has higher priority and will be used instead of the ID stored in /etc/machine-id.

Execution





Processes

https://en.wikipedia.org/wiki/Process_(computing) - an instance of a computer program that is being executed. It contains the program code and its current activity. Depending on the operating system (OS), a process may be made up of multiple threads of execution that execute instructions concurrently.

https://en.wikipedia.org/wiki/Process_management_(computing) - an integral part of any modern-day operating system (OS). The OS must allocate resources to processes, enable processes to share and exchange information, protect the resources of each process from other processes and enable synchronisation among processes. To meet these requirements, the OS must maintain a data structure for each process, which describes the state and resource ownership of that process, and which enables the OS to exert control over each process.













UPX: the Ultimate Packer for eXecutables - Homepage - a free, portable, extendable, high-performance executable packer for several executable formats. https://en.wikipedia.org/wiki/UPX - UPX uses a data compression algorithm called UCL, which is an open source implementation of portions of the proprietary NRV (Not Really Vanished) algorithm. UCL has been designed to be simple enough that a decompressor can be implemented in just a few hundred bytes of code. UCL requires no additional memory to be allocated for decompression, a considerable advantage that means that a UPX packed executable usually requires no additional memory. UPX (since 2.90 beta) can use LZMA on most platforms; however, this is disabled by default for 16-bit due to slow decompression speed on older computers (use --lzma to force it on). Starting with version 3.91, UPX also supports 64-Bit (x64) executable files on the Windows platform.[7] This feature is currently declared as experimental.







https://en.wikipedia.org/wiki/Process_control_block - also called Task Controlling Block, process table, Task Struct, or Switchframe, is a data structure in the operating system kernel containing the information needed to manage a particular process. The PCB is "the manifestation of a process in an operating system".





https://en.wikipedia.org/wiki/Process_identifier - (normally referred to as the process ID or just PID) is a number used by most operating system kernels — such as that of UNIX, Mac OS X or Microsoft Windows — to uniquely identify an active process. This number may be used as a parameter in various function calls, allowing processes to be manipulated, such as adjusting the process's priority or killing it altogether.

https://en.wikipedia.org/wiki/Process_group - In POSIX-conformant operating systems, a process group denotes a collection of one or more processes. Process groups are used to control the distribution of signals. A signal directed to a process group is delivered individually to all of the processes that are members of the group.

Process groups are themselves grouped into sessions. Process groups are not permitted to migrate from one session to another, and a process may only create new process groups belonging to the same session as it itself belongs to. Processes are not permitted to join process groups that are not in the same session as they themselves are.

New process images created by a call to a function of the exec family inherit the process group membership and the session membership of the old process image.





https://en.wikipedia.org/wiki/Orphan_process - a computer process whose parent process has finished or terminated, though it remains running itself.













https://github.com/gpakosz/whereami - Locate the current executable and the current module/library on the file system





BPF













Memory

System calls

See also Debugging#Tracing

https://en.wikipedia.org/wiki/System_call - how a program requests a service from an operating system's kernel. This may include hardware-related services (for example, accessing a hard disk drive), creation and execution of new processes, and communication with integral kernel services such as process scheduling. System calls provide an essential interface between a process and the operating system.









YouTube: Unix system calls (1/2) - Brian Will













/usr/include/sys/syscall.h /usr/include/bits/syscall.h









http://linux.die.net/man/2/intro - introduction to system calls





http://linux.die.net/man/2/syscalls - Linux system calls

System calls are generally not invoked directly, but rather via wrapper functions in glibc (or perhaps some other library). For details of direct invocation of a system call, see intro(2). Often, but not always, the name of the wrapper function is the same as the name of the system call that it invokes.





https://en.wikipedia.org/wiki/fork_(system_call) - an operation whereby a process creates a copy of itself. Fork is the primary (and historically, only) method of process creation on Unix-like operating systems.

https://en.wikipedia.org/wiki/exec_(system_call) - a functionality of an operating system that runs an executable file in the context of an already existing process, replacing the previous executable. This act is also referred to as an overlay. It is especially important in Unix-like systems, although exists elsewhere. As a new process is not created, the process identifier (PID) does not change, but the machine code, data, heap, and stack of the process are replaced by those of the new program. The exec is available for many programming languages including compilable languages and some scripting languages. In OS command interpreters, the exec built-in command replaces the shell process with the specified program.









kexec(8) - a system call that enables you to load and boot into another kernel from the currently running kernel. kexec performs the function of the boot loader from within the kernel. The primary difference between a standard system boot and a kexec boot is that the hardware initialization normally performed by the BIOS or firmware (depending on architecture) is not performed during a kexec boot. This has the effect of reducing the time required for a reboot.

















https://en.wikipedia.org/wiki/wait_(system_call) - suspends the execution of the parent process while the child executes. When the child process terminates, it returns an exit status to the operating system, which is then returned to the waiting parent process. The parent process then resumes execution.













https://en.wikipedia.org/wiki/epoll - a Linux kernel system call, a scalable I/O event notification mechanism, first introduced in Linux kernel 2.5.44. It is meant to replace the older POSIX select(2) and poll(2) system calls, to achieve better performance in more demanding applications, where the number of watched file descriptors is large (unlike the older system calls, which operate in O(n) time, epoll operates in O(1) time). epoll is similar to FreeBSD's kqueue, in that it operates on a configurable kernel object, exposed to user space as a file descriptor of its own.





https://en.wikipedia.org/wiki/sync_(Unix) - commits to disk all data in the kernel filesystem buffers, i.e., data which has been scheduled for writing via low-level I/O system calls. Note that higher-level I/O layers such as stdio may maintain separate buffers of their own.





https://github.com/stewartsmith/libeatmydata - An LD_PRELOAD library that disables all forms of writing data safely to disk. fsync() becomes a NO-OP, O_SYNC is removed etc. The idea is to use in testing to get faster test runs where real durability is not required. DO NOT use libeatmydata on software where you care about what it stores. It's called lib***EAT-MY-DATA*** for a reason.





https://en.wikipedia.org/wiki/select_(Unix) - for examining the status of file descriptors of open input/output channels. The select system call is similar to the poll facility introduced in UNIX System V and later operating systems.

https://en.wikipedia.org/wiki/splice_(system_call) - a Linux-specific system call that moves data between a file descriptor and a pipe without a round trip to user space. The related system call vmsplice() moves or copies data between a pipe and user space. Ideally, splice and vmsplice work by remapping pages and do not actually copy any data, which may improve I/O performance. As linear addresses do not necessarily correspond to contiguous physical addresses, this may not be possible in all cases and on all hardware combinations.

















https://github.com/majek/fluxcapacitor - a tool for making your program run without blocking on timeouts, on functions like poll and select, by spoofing POSIX time functions. It is somewhat similar to:FreezeGun for PythonTimeCop or DeLorean for Ruby. While these tools patch time libraries in Ruby and Python, fluxcapacitor works on a lower layer by "patching" low-level syscalls. That way, it can lie about time to any program in any programming language, as long as it runs on Linux.

Kernel interfaces





https://en.wikipedia.org/wiki/Application_binary_interface - the interface between two program modules, one of which is often a library or operating system, at the level of machine code. An ABI determines such details as how functions are called and in which binary format information should be passed from one program component to the next, or to the operating system in the case of a system call.

Adhering to ABIs (which may or may not be officially standardized) is usually the job of the compiler, OS or library writer, but application programmers may have to deal with ABIs directly when writing programs in a mix of programming languages, using foreign function call interfaces between them. ABIs differ from application programming interfaces (APIs), which similarly define interfaces between program components, but at the source code level.













http://linux-abi.sourceforge.net/ - The Linux abi is a patch to the linux kernel that allows a linux system to run foreign binaries. This was developed and written by Christoph Hellwig and Joerg Ahrens as a follow on to the iBCS/iBCS2 project written for the older 2.2.x kernel by Mike Jagdis.













Loader

https://en.wikipedia.org/wiki/Loader_(computing) - the part of an operating system that is responsible for loading programs and libraries. It is one of the essential stages in the process of starting a program, as it places programs into memory and prepares them for execution. Loading a program involves reading the contents of the executable file containing the program instructions into memory, and then carrying out other required preparatory tasks to prepare the executable for running. Once loading is complete, the operating system starts the program by passing control to the loaded program code.





https://en.wikipedia.org/wiki/Binfmt_misc - a capability of the Linux kernel which allows arbitrary executable file formats to be recognized and passed to certain user space applications, such as emulators and virtual machines. It is one of a number of binary format handlers in the kernel that are involved in preparing a user-space program to run.The executable formats are registered through the special purpose file system binfmt_misc file-system interface (usually mounted under part of /proc). This is either done directly by sending special sequences to the register procfs file or using a wrapper like Debian-based distributions binfmt-support package or systemd's systemd-binfmt.service

Formats

ELF

https://en.wikipedia.org/wiki/Executable_and_Linkable_Format - ELF, formerly called Extensible Linking Format, is a common standard file format for executables, object code, shared libraries, and core dumps. First published in the System V Release 4 (SVR4) Application Binary Interface (ABI) specification, and later in the Tool Interface Standard, it was quickly accepted among different vendors of Unix systems. In 1999 it was chosen as the standard binary file format for Unix and Unix-like systems on x86 by the 86open project. ELF is flexible and extensible by design, and it is not bound to any particular processor or architecture. This has allowed it to be adopted by many different operating systems on many different platforms.













https://github.com/vodik/pkgelf - A small tool for ELF-based packages to introspect dynamic linking.





Shared FLAT

bFLT

Binary Flat Format - uClinux uses a Binary Flat format commonly known as BFLT. It is a relatively simple and lightweight executable format based on the original a.out format. It has seen two major revisions, version 2 and more recently version 4. Version 2 is used by the m68k-coff compilers and is still in use with the ARM elf2flt converter while version 4 is used with the m68k elf2flt converter. Like many open source projects worked on by many individuals, little features have creped into versions without the revision field changing. As a consequence what we detail in each version may not strictly be the case in all circumstances. Both the 2.0.x and 2.4.x kernels’ bFLT loaders in the CVS support both formats. Earlier kernels may need to have binfmt_flat.c and flat.c patched to support version 4, should version 4 binaries report the following message when loading

FatELF

FatELF - a file format that embeds multiple ELF binaries for different architectures into one file. This is the Linux equivalent of what Mac OS X calls "Universal Binaries."The format is very simple: it adds some accounting info at the start of the file, and then appends all the ELF binaries after it, adding padding for alignment. The end of the file isn't touched, so you can still do things like self-extracting .zip files for multiple architectures with FatELF.FatELF lets you pack binaries into one file, seperated by OS ABI, OS ABI version, byte order and word size, and most importantly, CPU architecture.Work is focused on GNU/Linux, but this could be applied to most modern Unix systems: the BSDs, Solaris, etc.

αcτµαlly pδrταblε εxεcµταblε

Libraries

Program Library HOWTO - David A. Wheeler - This HOWTO for programmers discusses how to create and use program libraries on Linux. This includes static libraries, shared libraries, and dynamically loaded libraries.

Linux Journal: Linkers and Loaders

/lib # main library directory

/usr/lib # /lib can be a symlink to /usr/lib





Static

https://en.wikipedia.org/wiki/Static_library - or statically-linked library is a set of routines, external functions and variables which are resolved in a caller at compile-time and copied into a target application by a compiler, linker, or binder, producing an object file and a stand-alone executable. This executable and the process of compiling it are both known as a static build of the program. Historically, libraries could only be static. Static libraries are either merged with other static libraries and object files during building/linking to form a single executable, or they may be loaded at run-time into the address space of the loaded executable at a static memory offset determined at compile-time/link-time.

Dynamic

https://en.wikipedia.org/wiki/Dynamic_linker - the part of an operating system that loads and links the shared libraries needed by an executable when it is executed (at "run time"), by copying the content of libraries from persistent storage to RAM, and filling jump tables and relocating pointers. The specific operating system and executable format determine how the dynamic linker functions and how it is implemented.

Linking is often referred to as a process that is performed when the executable is compiled, while a dynamic linker is a special part of an operating system that loads external shared libraries into a running process and then binds those shared libraries dynamically to the running process. This approach is also called dynamic linking or late linking.

dlfcn.h - dynamic linking





https://en.wikipedia.org/wiki/Dynamic_loading - a mechanism by which a computer program can, at run time, load a library (or other binary) into memory, retrieve the addresses of functions and variables contained in the library, execute those functions or access those variables, and unload the library from memory. Unlike static linking and dynamic linking, dynamic loading allows a computer program to start up in the absence of these libraries, to discover available libraries, and to potentially gain additional functionality.









http://cateee.net/lkddb/web-lkddb/USELIB.html - This option enables the uselib syscall, a system call used in the dynamic linker from libc5 and earlier. glibc does not use this system call. If you intend to run programs built on libc5 or earlier, you may need to enable this syscall. Current systems running glibc can safely disable this.

http://linux.die.net/man/2/uselib - system call uselib() serves to load a shared library to be used by the calling process. It is given a pathname. The address where to load is found in the library itself. The library can have any recognized binary format.





The programs ld.so and ld-linux.so* find and load the shared libraries needed by a program, prepare the program to run, and then run it.

Linux binaries require dynamic linking (linking at run time) unless the -static option was given to ld(1) during compilation.

The program ld.so handles a.out binaries, a format used long ago; ld-linux.so* handles ELF (/lib/ld-linux.so.1 for libc5, /lib/ld-linux.so.2 for glibc2), which everybody has been using for years now. Otherwise both have the same behavior, and use the same support files and programs ldd(1), ldconfig(8) and /etc/ld.so.conf.













ldconfig - configure dynamic linker run-time bindings

http://linux.die.net/man/1/ldd - prints the shared libraries required by each program or shared library specified on the command line.

ldd /usr/bin/ls

linux-vdso.so.1 (0x00007ffd9afde000) libcap.so.2 => /usr/lib/libcap.so.2 (0x00007f35e72f9000) libc.so.6 => /usr/lib/libc.so.6 (0x00007f35e6f58000) /lib64/ld-linux-x86-64.so.2 (0x00007f35e74fd000)





"Linux puts you in full control. This is not always seen from everyone’s perspective, but a power user loves to be in control. ... what happens when we set up the environmental variable LD_PRELOAD? This variable forces some libraries to be loaded for a program. In our case, it loads unrandom.so for random_num, even though the program itself does not ask for it. The following command may be interesting:"

$ LD_PRELOAD=$PWD/unrandom.so ldd random_nums linux-vdso.so.1 => (0x00007fff369dc000) /some/path/to/unrandom.so (0x00007f262b439000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f262b044000) /lib64/ld-linux-x86-64.so.2 (0x00007f262b63d000)





https://www.flamingspork.com/projects/libeatmydata - An LD_PRELOAD library that disables all forms of writing data safely to disk. fsync() becomes a NO-OP, O_SYNC is removed etc. The idea is to use in testing to get faster test runs where real durability is not required. DO NOT use libeatmydata on software where you care about what it stores. It's called lib***EAT-MY-DATA*** for a reason https://github.com/stewartsmith/libeatmydata







https://github.com/wolfcw/libfaketime - libfaketime intercepts various system calls that programs use to retrieve the current date and time. It then reports modified (faked) dates and times (as specified by you, the user) to these programs. This means you can modify the system time a program sees without having to change the time system-wide.





https://github.com/DavidBuchanan314/TARDIS - Trace And Rewrite Delays In Syscalls: Hooking time-related Linux syscalls to warp a process's perspective of time, using ptrace.





https://github.com/riboseinc/retrace - a versatile security vulnerability / bug discovery tool through monitoring and modifying the behavior of compiled binaries on Linux, OpenBSD/FreeBSD/NetBSD (shared object) and macOS (dynamic library).





https://github.com/SimonKagstrom/kcov - a Linux/OSX code coverage tester for compiled languages, Python and Bash. Kcov was originally a fork of Bcov, but has since evolved to support a large feature set in addition to that of Bcov. Kcov, like Bcov, uses DWARF debugging information for compiled programs to make it possible to collect coverage information without special compiler switches.





https://github.com/sickill/stderred - stderr in red. stderred hooks on write() and a family of stream functions (fwrite, fprintf, error...) from libc in order to colorize all stderr output that goes to terminal thus making it distinguishable from stdout. Basically it wraps text that goes to file with descriptor "2" with proper ANSI escape codes making text red.





https://github.com/zardus/preeny - helps you pwn noobs by making it easier to interact with services locally. It disables fork(), rand(), and alarm() and, if you want, can convert a server application to a console one using clever/hackish tricks, and can even patch binaries!





https://elinux.org/Electric_Fence - Electric Fence helps you detect two common programming bugs: software that overruns the boundaries of a malloc()memory allocation, software that touches a memory allocation that has been released by free(). Unlike other malloc() debuggers, Electric Fence will detect read accesses as well as writes, and it will pinpoint the exact instruction that causes an error. Electric Fence uses the virtual memory hardware of your computer to place an inaccessible memory page immediately after (or before, at the user's option) each memory allocation.





https://github.com/dex4er/fakechroot - fakechroot runs a command in an environment were is additional possibility to use chroot(8) command without root privileges. This is useful for allowing users to create own chrooted environment with possibility to install another packages without need for root privileges. fakechroot replaces some libc library functions (chroot(2), open(2), etc.) by ones that simulate the effect of being called with root privileges. These wrapper functions are in a shared library libfakechroot.so which is loaded through the LD_PRELOAD mechanism of the dynamic loader. (See ld.so(8))

https://blitiri.com.ar/p/libfiu - libfiu is a C library for fault injection. It aims to simplify automated testing of software behaviour at failure scenarios, for software which is supposed to be fault-resistant or at least gracefully cope with failures. It provides functions to mark "points of failure" inside your code (the core API), and functions to enable/disable the failure of those points (the control API).

https://github.com/paultag/tmperamental - a library that intercepts filesystem writes, and causes loud failures when writes are attempted on /tmp/*. The idea here is by setting TMPDIR and friends, you are able to use libtmperamental to catch programs in the act of not respecting such env-vars.

https://github.com/mirrorer/afl/tree/master/libdislocator - a companion library that can be used as a drop-in replacement for the libc allocator in the fuzzed binaries. It improves the odds of bumping into heap-related security bugs in several ways.

vDSO

https://en.wikipedia.org/wiki/vDSO - virtual dynamically linked shared object) is a Linux kernel mechanism for exporting a carefully selected set of kernel space routines to user space applications so that applications can call these kernel space routines in-process, without incurring the performance penalty of a context switch that is inherent when calling these same kernel space routines by means of the system call interface. vDSO has been developed to offer the vsyscall features and to overcome its limitations: small amount of allocated memory allowing only 4 system calls and same addresses ABI in each process compromising security. This security issue has been by emulating a virtual system call introducing a penalty latency.

a.out

https://en.wikipedia.org/wiki/a.out - a file format used in older versions of Unix-like computer operating systems for executables, object code, and, in later systems, shared libraries. The name stands for "assembler output", and was coined by Ken Thompson as the fixed name for output of his PDP-7 assembler in 1968. The term was subsequently applied to the format of the resulting file, to contrast with other formats for object code.

"a.out" remains the default output file name for executables created by certain compilers and linkers when no output name is specified, even though the created files actually are not in the a.out format.

https://blackfin.uclinux.org/doku.php?id=toolchain:executable_file_formats





Memory

See also Computing#Memory









ps_mem - A utility to accurately report the core memory usage for a program





Paging









numa(7) - Linux manual page - overview of Non-Uniform Memory Architecture

https://en.wikipedia.org/wiki/Non-uniform_memory_access - a computer memory design used in multiprocessing, where the memory access time depends on the memory location relative to the processor. Under NUMA, a processor can access its own local memory faster than non-local memory (memory local to another processor or memory shared between processors). The benefits of NUMA are limited to particular workloads, notably on servers where the data is often associated strongly with certain tasks or users.





Co-ordination

https://en.wikipedia.org/wiki/Synchronization_(computer_science) - refers to one of two distinct but related concepts: synchronization of processes, and synchronization of data. Process synchronization refers to the idea that multiple processes are to join up or handshake at a certain point, in order to reach an agreement or commit to a certain sequence of action. Data synchronization refers to the idea of keeping multiple copies of a dataset in coherence with one another, or to maintain data integrity. Process synchronization primitives are commonly used to implement data synchronization.





RCU

LWN.net: What is RCU, Fundamentally? - Read-copy update (RCU) is a synchronization mechanism that was added to the Linux kernel in October of 2002. RCU achieves scalability improvements by allowing reads to occur concurrently with updates. In contrast with conventional locking primitives that ensure mutual exclusion among concurrent threads regardless of whether they be readers or updaters, or with reader-writer locks that allow concurrent reads but not in the presence of updates, RCU supports concurrency between a single updater and multiple readers. RCU ensures that reads are coherent by maintaining multiple versions of objects and ensuring that they are not freed up until all pre-existing read-side critical sections complete. RCU defines and uses efficient and scalable mechanisms for publishing and reading new versions of an object, and also for deferring the collection of old versions. These mechanisms distribute the work among read and update paths in such a way as to make read paths extremely fast. In some cases (non-preemptable kernels), RCU's read-side primitives have zero overhead.









https://en.wikipedia.org/wiki/Read-copy-update - a synchronization mechanism based on mutual exclusion.[note 1] It is used when performance of reads is crucial and is an example of space–time tradeoff, enabling fast operations at the cost of more space.Read-copy-update allows multiple threads to efficiently read from shared memory by deferring updates after pre-existing reads to a later time while simultaneously marking the data, ensuring new readers will read the updated data. This makes all readers proceed as if there were no synchronization involved, hence they will be fast, but also making updates more difficult.

FPU

Inter-process communication

https://en.wikipedia.org/wiki/Inter-process_communication - the activity of sharing data across multiple and commonly specialized processes using communication protocols. Typically, applications using IPC are categorized as clients and servers, where the client requests data and the server responds to client requests. Many applications are both clients and servers, as commonly seen in distributed computing. Methods for achieving IPC are divided into categories which vary based on software requirements, such as performance and modularity requirements, and system circumstances, such as network bandwidth and latency.





Interrupts

The Linux kernel provides the following IPC mechanisms: Signals, Anonymous Pipes, Named Pipes or FIFOs, SysV Message Queues, POSIX Message Queues, SysV Shared memory, POSIX Shared memory, SysV semaphores, POSIX semaphores, FUTEX locks, File-backed and anonymous shared memory using mmap, UNIX Domain Sockets, Netlink Sockets, Network Sockets, Inotify mechanisms, FUSE subsystem, D-Bus subsystem.

Interrupts can be viewed as a mean of communication between the CPU and the OS kernel. Signals can be viewed as a mean of communication between the OS kernel and OS processes.

Interrupts may be initiated by the CPU (exceptions - e.g.: divide by zero, page fault), devices (hardware interrupts - e.g: input available), or by a CPU instruction (traps - e.g: syscalls, breakpoints). They are eventually managed by the CPU, which "interrupts" the current task, and invokes an OS-kernel provided ISR/interrupt handler.

Signals may be initiated by the OS kernel (e.g: SIGFPE, SIGSEGV, SIGIO), or by a process(kill()). They are eventually managed by the OS kernel, which delivers them to the target thread/process, invoking either a generic action (ignore, terminate, terminate and dump core) or a process-provided signal handler. [50]









Shared memory

SysV

POSIX

http://linux.die.net/man/7/shm_overview - overview of POSIX shared memory API allows processes to communicate information by sharing a region of memory.

Unix signals

https://en.wikipedia.org/wiki/Unix_signal - a limited form of inter-process communication used in Unix, Unix-like, and other POSIX-compliant operating systems. A signal is an asynchronous notification sent to a process or to a specific thread within the same process in order to notify it of an event that occurred. Signals originated in 1970s Bell Labs Unix and have been more recently specified in the POSIX standard.

When a signal is sent, the operating system interrupts the target process' normal flow of execution to deliver the signal. Execution can be interrupted during any non-atomic instruction. If the process has previously registered a signal handler, that routine is executed. Otherwise, the default signal handler is executed.





A signal is a software interrupt delivered to a process. The operating system uses signals to report exceptional situations to an executing program. Some signals report errors such as references to invalid memory addresses; others report asynchronous events, such as disconnection of a phone line.

https://en.wikipedia.org/wiki/Sigaction - a function API defined by POSIX to give the programmer access to what should be a program's behavior when receiving specific OS signals.









Unix domain sockets









Pipes

If |& is used, the standard error of command is connected to command2's standard input through the pipe; it is shorthand for 2>&1 |.

















https://github.com/rom1v/delay - a shell command introducing a constant delay between its standard input and its standard output.

Socketpipe - directly connects the input/output of a remote process with local programs over a TCP/IP socket. Although the initial communication setup is performed through client-server intermediaries such as SSH or RSH, the communication channel that socketpipe establishes is a direct socket connection between the local and the remote processes, similar to a local pipe. The use of socketpipe removes the inefficiency of multiple data copies and context switches through RSH/SSH, and can in many cases offer dramatic throughput improvements.

Spiped - pronounced "ess-pipe-dee", is a utility for creating symmetrically encrypted and authenticated pipes between socket addresses, so that one may connect to one address (e.g., a UNIX socket on localhost) and transparently have a connection established to another address (e.g., a UNIX socket on a different system). This is similar to 'ssh -L' functionality, but does not use SSH and requires a pre-shared symmetric key.

pv

pv - Pipe Viewer, a terminal-based tool for monitoring the progress of data through a pipeline. It can be inserted into any normal pipeline between two processes to give a visual indication of how quickly data is passing through, how long it has taken, how near to completion it is, and an estimate of how long it will be until completion.

pv source > target # cp with progress bar tar czf images.tar.gz image1 image2 image3 | pv > target # tar pv source | gzip > target.gz # or gzip < source | pv > target.gz # gzip -L 25k # rate limit -d PID[:FD], --watchfd PID[:FD] # watch existing process









Pipeglade - Graphical User Interfaces, The UNIX Way, a helper program that displays graphical user interfaces for other programs. It renders the GUI definition found in a GtkBuilder file (created using the Glade Interface Designer), and communicates with the main program solely via pipes or fifos. [58]





https://github.com/jnweiger/showspeed - Print I/O activity of process, files, or network. Print estimated time of arrival.





https://github.com/roddhjav/progressbar - A pacman like progress bar in bash.

Named pipes





D-Bus

D-Bus - a message bus system, a simple way for applications to talk to one another. In addition to interprocess communication, D-Bus helps coordinate process lifecycle; it makes it simple and reliable to code a "single instance" application or daemon, and to launch applications and daemons on demand when their services are needed.

D-Bus supplies both a system daemon (for events such as "new hardware device added" or "printer queue changed") and a per-user-login-session daemon (for general IPC needs among user applications). Also, the message bus is built on top of a general one-to-one message passing framework, which can be used by any two apps to communicate directly (without going through the message bus daemon). Currently the communicating applications are on one computer, or through unencrypted TCP/IP suitable for use behind a firewall with shared NFS home directories. (Help wanted with better remote transports - the transport mechanism is well-abstracted and extensible.)

https://wiki.archlinux.org/index.php/D-Bus - D-Bus is enabled automatically when using systemd because dbus is a dependency of systemd.









kdbus

https://github.com/gregkh/kdbus - not in kernel, give it a few years

http://lwn.net/Articles/580194/ - "kdbus, an in-kernel implementation of D-Bus. This implementation is able to carry large amounts of data; it can be reasonably used for gigabyte-sized message streams. It can perform zero-copy message passing, but even in the worst case, a message and its response are passed with no more than two copy operations, two validations, and two context switches. Full credential information (user ID, process ID, SELinux label, control group information, capabilities, and much more) is passed with each message, and all messages carry timestamps. Kdbus is always available to the system (no need to wait for the D-Bus daemon to be started), Linux security modules can hook into it directly, various race conditions have been fixed, and the API has simplified."

VARLINK

VARLINK - an interface description format and protocol that aims to make services accessible to both humans and machines in the simplest feasible way.A varlink interface combines the classic UNIX command line options, STDIN/OUT/ERROR text formats, man pages, service metadata and provides the equivalent over a single file descriptor, a.k.a. “FD3”.Varlink is plain-text, type-safe, discoverable, self-documenting, remotable, testable, easy to debug. Varlink is accessible from any programming environment. [59]

Utils

at bg chroot fg time





https://gist.github.com/anonymous/7529780 - script to show swapped processes





kill

In Unix and Unix-like operating systems, kill is a command used to send a signal to a process. By default, the message sent is the termination signal, which requests that the process exit. But kill is something of a misnomer; the signal sent may have nothing to do with process killing. The kill command is a wrapper around the kill() system call, which sends signals to processes or process groups on the system, referenced by their numeric process IDs (PIDs) or process group IDs (PGIDs). kill is always provided as a standalone utility as defined by the POSIX standard. However, most shells have built-in kill commands that may slightly differ from it.

A process can be sent a SIGTERM signal in four ways (the process ID is '1234' in this case):

kill 1234 kill -s TERM 1234 kill -TERM 1234 kill -15 1234

The process can be sent a SIGKILL signal in three ways:

kill -s KILL 1234 kill -KILL 1234 kill -9 1234

killall -u username terminate all username's applications to log them out

http://psdoom.sourceforge.net/ - kill processes in doom

nohup

nohup - Run a command immune to hangups, runs the given command with hangup signals ignored, so that the command can continue running in the background after you log out.

reptyr

https://github.com/nelhage/reptyr - a utility for taking an existing running program and attaching it to a new terminal. Started a long-running process over ssh, but have to leave and don't want to interrupt it? Just start a screen, use reptyr to grab it, and then kill the ssh session and head on home.

Debugging

See also Development#Debugging





Scheduling





https://en.wikipedia.org/wiki/O(1)_scheduler - a kernel scheduling design that can schedule processes within a constant amount of time, regardless of how many processes are running on the operating system. One of the major goals of operating system designers is to minimize overhead and jitter of OS services, so that application programmers who use them endure less of a performance impact. O(1) scheduler providing "constant time" scheduling services has helped in this regard.









nice

nice - run a program with modified scheduling priority https://en.wikipedia.org/wiki/Nice_(Unix)



nice -n 19 [command] run a command at lowest priority

nice -n -20 [command] run a command at highest priority

VeryNice

ionice

https://linux.die.net/man/1/ionice - sets or gets the io scheduling class and priority for a program. If no arguments or just -p is given, ionice will query the current io scheduling class and priority for that process.

Ananicy

https://github.com/Nefelim4ag/Ananicy - Another auto nice daemon, with community rules support (Use pull request please)

Threads

https://en.wikipedia.org/wiki/Thread_(computer_science) - a thread of execution is the smallest sequence of programmed instructions that can be managed independently by a scheduler, which is typically a part of the operating system. The implementation of threads and processes differs between operating systems, but in most cases a thread is a component of a process. Multiple threads can exist within one process, executing concurrently (one starting before others finish) and share resources such as memory, while different processes do not share these resources. In particular, the threads of a process share its instructions (executable code) and its context (the values of its variables at any given time).

https://en.wikipedia.org/wiki/Green_threads - threads that are scheduled by a runtime library or virtual machine (VM) instead of natively by the underlying operating system. Green threads emulate multithreaded environments without relying on any native OS capabilities, and they are managed in user space instead of kernel space, enabling them to work in environments that do not have native thread support.









"If you want to run code on many platforms, go for Posix Threads. They are available almost everywhere and are quite mature. OTOH if you only use Linux/gcc std::thread is perfectly fine - it has a higher abstraction level, a really good interface and plays nicely with other C++11 classes.

"The C++11 std::thread class unfortunately doesn't work reliably (yet) on every platform, even if C++11 seems available. For instance in native Android std::thread or Win64 it just does not work or has severe performance bottlenecks (as of 2012).

"A good replacement is boost::thread - it is very similar to std::thread (actually it is from the same author) and works reliably, but, of course, it introduces another dependency from a third party library."

"The std::thread library is implemented on top of pthreads in an environment supporting pthreads (for example: libstdc++).

"I think the big difference between the two is abstraction. std::thread is a C++ class library. The std::thread library includes many abstract features, for example: scoped locks, recursive mutexes, future/promise design pattern implementations, and more."





CPU scaling

https://en.wikipedia.org/wiki/Dynamic_frequency_scaling - also known as CPU throttling) is a technique in computer architecture whereby the frequency of a microprocessor can be automatically adjusted "on the fly" depending on the actual needs, to conserve power and reduce the amount of heat generated by the chip. Dynamic frequency scaling helps preserve battery on mobile devices and decrease cooling cost and noise on quiet computing settings, or can be useful as a security measure for overheated systems (e.g. after poor overclocking). Dynamic frequency scaling is used in all ranges of computing systems, ranging from mobile systems to data centers to reduce the power at the times of low workload.





cat /sys/devices/system/cpu/cpufreq/policy*/scaling_available_governors

Watchdog





Security





cgroups

Control Groups - provide a mechanism for aggregating/partitioning sets of tasks, and all their future children, into hierarchical groups with specialized behaviour.





Capabilities

https://en.wikipedia.org/wiki/Capability-based_security - a concept in the design of secure computing systems, one of the existing security models. A capability (known in some systems as a key) is a communicable, unforgeable token of authority. It refers to a value that references an object along with an associated set of access rights. A user program on a capability-based operating system must use a capability to access an object. Capability-based security refers to the principle of designing user programs such that they directly share capabilities with each other according to the principle of least privilege, and to the operating system infrastructure necessary to make such transactions efficient and secure. Capability-based security is to be contrasted with an approach that uses hierarchical protection domains.

Linux manual page: capabilities(7) - For the purpose of performing permission checks, traditional UNIX implementations distinguish two categories of processes: privileged processes (whose effective user ID is 0, referred to as superuser or root), and unprivileged processes (whose effective UID is nonzero). Privileged processes bypass all kernel permission checks, while unprivileged processes are subject to full permission checking based on the process's credentials (usually: effective UID, effective GID, and supplementary group list). Starting with kernel 2.2, Linux divides the privileges traditionally associated with superuser into distinct units, known as capabilities, which can be independently enabled and disabled. Capabilities are a per-thread attribute.

Process Accounting

Process Accounting - a security method in which an administrator may keep track of system resources used and their allocation among users, provide for system monitoring, and minimally track a user's commands.Process accounting has both positive and negative points. One of the positives is that an intrusion may be narrowed down to the point of entry. A negative is the amount of logs generated by process accounting, and the disk space they may require. This section walks an administrator through the basics of process accounting.

Boot





http://en.wikipedia.org/wiki/Linux_startup_process - the multi-stage initialization process performed during booting a Linux installation. It is in many ways similar to the BSD and other Unix-style boot processes, from which it derives.

Booting a Linux installation involves multiple stages and software components, including firmware initialization, execution of a boot loader, loading and startup of a Linux kernel image, and execution of various startup scripts and daemons. For each of these stages and components there are different variations and approaches; for example, GRUB, LILO, SYSLINUX or Loadlin can be used as boot loaders, while the startup scripts can be either traditional init-style, or the system configuration can be performed through more modern alternatives such as systemd or Upstart.









IBM: Inside the Linux boot process - Take a guided tour from the Master Boot Record to the first user-space application









e4rat - reduce boot time (into X) by some 50% for ext4

BIOS / UEFI

https://en.wikipedia.org/wiki/BIOS - an acronym for Basic Input/Output System and also known as the System BIOS, ROM BIOS or PC BIOS) is a type of firmware used to perform hardware initialization during the booting process (power-on startup) on IBM PC compatible computers, and to provide runtime services for operating systems and programs. The BIOS firmware is built into personal computers (PCs), and it is the first software they run when powered on. The name itself originates from the Basic Input/Output System used in the CP/M operating system in 1975. Originally proprietary to the IBM PC, the BIOS has been reverse engineered by companies looking to create compatible systems and the interface of that original system serves as a de facto standard.

The fundamental purposes of the BIOS in modern PCs are to initialize and test the system hardware components, and to load a boot loader or an operating system from a mass memory device. The BIOS additionally provides an abstraction layer for the hardware, i.e., a consistent way for application programs and operating systems to interact with the keyboard, display, and other input/output (I/O) devices. Variations in the system hardware are hidden by the BIOS from programs that use BIOS services instead of directly accessing the hardware. MS-DOS (PC DOS), which was the dominant PC operating system from the early 1980s until the mid-1990s, relied on BIOS services for disk, keyboard, and text display functions. MS Windows NT, Linux, and other protected mode operating systems in general ignore the abstraction layer provided by the BIOS and do not use it after loading, instead accessing the hardware components directly.

Most BIOS implementations are specifically designed to work with a particular computer or motherboard model, by interfacing with various devices that make up the complementary system chipset. Originally, BIOS firmware was stored in a ROM chip on the PC motherboard; in modern computer systems, the BIOS contents are stored on flash memory so it can be rewritten without removing the chip from the motherboard. This allows easy updates to the BIOS firmware so new features can be added or bugs can be fixed, but it also creates a possibility for the computer to become infected with BIOS rootkits.

https://en.wikipedia.org/wiki/Power-on_self-test - POST routines are part of a device's pre-boot sequence and only once they complete successfully is the bootstrap loader code invoked to load an operating system





https://github.com/luke8086/boot2c - A simple way to write standalone C programs for i386





https://en.wikipedia.org/wiki/Unified_Extensible_Firmware_Interface - UEFI, was designed as a successor to BIOS, aiming to address its technical shortcomings. As of 2014, new PC hardware predominantly ships with UEFI firmware.

https://en.wikipedia.org/wiki/Unified_EFI_Forum or UEFI Forum is an alliance between several leading technology companies to modernize the booting process. The board of directors includes representatives from eleven "Promoter" companies: AMD, American Megatrends, Apple, Dell, HP, IBM, Insyde Software, Intel, Lenovo, Microsoft, and Phoenix Technologies.









ACPI

https://en.wikipedia.org/wiki/Advanced_Configuration_and_Power_Interface - ACPI - specification provides an open standard that the operating systems can use for computer hardware discovery, configuration, power management, and monitoring. Internally, ACPI exports the available functionalities by providing certain instruction lists as part of the system firmware, which the operating system kernel interprets and executes to perform desired operations, using a form of embedded virtual machine.

"Overall design decision was not without criticism. In November 2003, Linus Torvalds—author of the Linux kernel—described ACPI as "a complete design disaster in every way". In 2001, other senior Linux software developers like Alan Cox expressed concerns about the requirements that bytecode from an external source must be run by the kernel with full privileges, as well as the overall complexity of the ACPI specification. In 2014, Mark Shuttleworth, founder of the Ubuntu Linux distribution, compared ACPI with Trojan horses."





iasl - ACPI Source Language compiler/decompiler









Coreboot

https://en.wikipedia.org/wiki/Coreboot - formerly known as LinuxBIOS, is a free software project aimed at replacing proprietary firmware (BIOS or UEFI) found in most computers with a lightweight firmware designed to perform only the minimum number of tasks necessary to load and run a modern 32-bit or 64-bit operating system.

Since coreboot initializes the bare hardware, it must be ported to every chipset and motherboard that it supports. As a result, coreboot is available only for a limited number of hardware platforms and motherboard models.

Network boot





https://en.wikipedia.org/wiki/Preboot_Execution_Environment - PXE, sometimes pronounced as pixie, a specification that describes a standardized client-server environment that boots a software assembly, retrieved from a network, on PXE-enabled clients. On the client side it requires only a PXE-capable network interface controller (NIC), and uses a small set of industry-standard network protocols such as DHCP and TFTP.

The concept behind the PXE originated in the early days of protocols like BOOTP/DHCP/TFTP, and as of 2015 it forms part of the Unified Extensible Firmware Interface (UEFI) standard. Given fast and reliable local area networks (LANs), PXE is the most frequent choice for operating system booting, installation and deployment.





netboot.xyz - a way to select various operating system installers or utilities from one place within the BIOS without the need of having to go retrieve the media to run the tool. iPXE is used to provide a user friendly menu from within the BIOS that lets you easily choose the operating system you want along with any specific types of versions or bootable flags. You can remote attach the ISO to servers, set it up as a rescue option in Grub, or even set up your home network to boot to it by default so that it's always available. [72] https://github.com/antonym/netboot.xyz







PXELINUX - a Syslinux derivative, for booting from a network server using a network ROM conforming to the Intel PXE (Pre-Execution Environment) specification. PXELINUX is not a program intended to be flashed or burned into a PROM on the network card. For such possibility, check out iPXE (http://ipxe.org/). If you want to create PXE-compliant boot PROM for your network card (to use with PXELINUX, for example), check out NetBoot (http://netboot.sourceforge.net/).





Netboot - tools for network boot, allows to remote boot a computer over an IP network without access to a hard disk or a diskette. It is therefore ideally suited for diskless or thin clients using an x86 processor. Netboot is open source software licensed by the GNU GPL, and contains all necessary tools including a PXE compliant bootrom. Operating systems supported on the diskless client are Linux, FreeBSD and many DOS variants.





iPXE - the leading open source network boot firmware. It provides a full PXE implementation enhanced with additional features such as: boot from a web server via HTTP, boot from an iSCSI SAN, boot from a Fibre Channel SAN via FCoE, boot from an AoE SAN, boot from a wireless network, boot from a wide-area network, boot from an Infiniband network, control the boot process with a script, You can use iPXE to replace the existing PXE ROM on your network card, or you can chainload into iPXE to obtain the features of iPXE without the hassle of reflashing. iPXE is free, open-source software licensed under the GNU GPL (with some portions under GPL-compatible licences), and is included in products from several network card manufacturers and OEMs. https://en.wikipedia.org/wiki/iPXE - an open-source implementation of the Preboot eXecution Environment (PXE) client firmware and bootloader, created in 2010 as a fork of gPXE. It can be used to enable computers without built-in PXE support to boot from the network, or to extend an existing PXE client implementation so it supports additional protocols. While standardized PXE clients use TFTP to transfer data, non-standardized iPXE client firmware adds the ability to retrieve data through other protocols, including HTTP, iSCSI, ATA over Ethernet (AoE), and Fibre Channel over Ethernet (FCoE). Also, on supported hardware iPXE firmware can use a Wi-Fi link rather than requiring a wired connection. iPXE firmware cannot be considered as a "drop-in" replacement for PXE firmware.







Etherboot/gPXE - an open source (GPL) network bootloader. It provides a direct replacement for proprietary PXE ROMs, with many extra features such as DNS, HTTP, iSCSI, etc. https://github.com/etherboot/gpxe https://en.wikipedia.org/wiki/GPXE







ERPXE - a complete PXE solution featuring a broad range of recovery tools and various OS installations in one box.

ERPXE is a free and open source project, which uses the GNU General Public License version 3.









https://en.wikipedia.org/wiki/Remote_Initial_Program_Load - a protocol for starting a computer and loading its operating system from server via a network. Such a server runs a network operating system such as LAN Manager, LAN Server, Windows NT Server, Novell NetWare, Solaris or Linux. RIPL is similar to Preboot Execution Environment (PXE), but it uses the Novell NetWare based boot method. It was originally developed by IBM.









https://en.wikipedia.org/wiki/Wake-on-LAN - an Ethernet or Token ring computer networking standard that allows a computer to be turned on or awakened by a network message.

Boot sector

Boot loader

"In order to boot Arch Linux, a Linux-capable boot loader such as GRUB(2), Syslinux, LILO or GRUB Legacy must be installed to the Master Boot Record or the GUID Partition Table. The boot loader is responsible for loading the kernel and initial ramdisk before initiating the boot process."

loadlin

https://en.wikipedia.org/wiki/loadlin - a Linux boot loader that runs under DOS or Microsoft Windows (95, 98 or Me only). It allows the Linux system to load and replace the running DOS/Windows without altering existing DOS/Windows system files. Due to its structure, loadlin only works on DOS-based operating systems, and will not work on NT-based versions of Windows.

LILO

https://en.wikipedia.org/wiki/LILO_(boot_loader) - a boot loader for Linux and was the default boot loader for most Linux distributions in the years after the popularity of loadlin. Today, most distributions use GRUB as the default boot loader. LILO has been discontinued in December 2015, with a request for potential developers.

GRUB

grub-mkconfig -o /boot/grub/grub.cfg # regenerate config after changing kernels installed





https://gna.org/projects/grub4dos/ - for multibooting via a FAT filesystem

Syslinux





https://github.com/GNU-Pony/splashtool - A simple tool for creating SYSLINUX splashes without fuss

rEFInd

rEFInd - fork of the rEFIt boot manager for computers based on the Extensible Firmware Interface (EFI) and Unified EFI (UEFI) https://github.com/EvanPurkhiser/rEFInd-minimal-theme



Plop Boot Manager(s)

Smart BootManager

SeaBIOS

SeaBIOS - an open source implementation of a 16bit X86 BIOS. SeaBIOS can run in an emulator or it can run natively on X86 hardware with the use of coreboot.

U-Boot

U-Boot - a boot loader for Embedded boards based on PowerPC, ARM, MIPS and several other processors, which can be installed in a boot ROM and used to initialize and test the hardware or to download and run application code. The development of U-Boot is closely related to Linux: some parts ofthe source code originate in the Linux source tree, we have some header files in common, and special provision has been made to support booting of Linux images. https://gitlab.denx.de/u-boot/u-boot https://github.com/hardkernel/u-boot https://en.wikipedia.org/wiki/Das_U-Boot



Petitboot

Petitboot - a platform independent bootloader based on the Linux kexec warm reboot mechanism. Petitboot supports loading kernel, initrd and device tree files from any Linux mountable filesystem, plus can load files from the network using the FTP, SFTP, TFTP, NFS, HTTP and HTTPS protocols. Petitboot can boot any operating system that includes kexec boot support.Petitboot includes an ncurses based user interface program. This user interface program can be used to boot the system remotely via telnet or ssh sessions. Multiple user interface programs can run simultaneously.Petitboot is licensed under the GPLv2. https://github.com/open-power/petitboot



Petitboot has a client-server design. The petitboot server, pb-discover, uses several event mechanisms to dynamically update its knowledge of available boot configurations as they become available or unavailable at run time. The petitboot client programs implement the petitboot user interfaces and display the curent boot configurations known by pb-discover.

pb-discover exposes a generic event mechanism that allows utility programs to provide boot configuration information. Petitboot includes utility programs to process boot configuration info from DHCP servers, and if configured to do so, will manage any boot configuration info received.





petitboot - original homepage













zectl

Initial ramdisk

initrd

initrd (initial ramdisk) is a scheme for loading a temporary root file system into memory in the boot process of the Linux kernel. initrd and initramfs refer to two different methods of achieving this. Both are commonly used to make preparations before the real root file system can be mounted.

initramfs

https://en.wikipedia.org/wiki/initramfs - abbreviated from "initial RAM file system", is the successor of initrd ("initial ramdisk"). It is a cpio ("copy in and out") archive of the initial file system that gets loaded into memory during the Linux startup process. The Linux kernel copies the contents of the archive into the built-in rootfs file-system-like infrastructure (which is either based on ramfs, or the more regulated tmpfs), and then attempts to run the resulting /init program, before falling back on older methods of initialization; the init program is meant to complete certain tasks before the "real" or final root file system is mounted over rootfs. Thus, initramfs needs to contain all of the device drivers and tools needed to mount the final root file system.





















Mkinitcpio

https://wiki.archlinux.org/index.php/Mkinitcpio - next generation of initramfs creation.

ps -p 1 -o comm=

Dracut

distri

u-root

u-root - an embeddable root file system intended to be placed in a flash device as part of the firmware image, along with a Linux kernel. Unlike most embedded root file systems, which consist of large binaries, u-root only has five: an init program and four Go compiler binaries. https://github.com/u-root/u-root



Kernal loading

https://en.wikipedia.org/wiki/Multiboot_Specification - an open standard describing how a boot loader can load an x86 operating system kernel.[1][2] The specification allows any compliant boot loader implementation to boot any compliant operating system kernel.

https://en.wikipedia.org/wiki/vmlinux - a statically linked executable file that contains the Linux kernel in one of the object file formats supported by Linux, which includes ELF, COFF and a.out. The vmlinux file might be required for kernel debugging, symbol table generation or other operations, but must be made bootable before being used as an operating system kernel by adding a multiboot header, bootsector and setup routines.

https://en.wikipedia.org/wiki/Mode_setting - a software operation that activates a display mode (screen resolution, color depth, and refresh rate) for a computer's display controller.

https://wiki.archlinux.org/index.php/kernel_mode_setting - a method for setting display resolution and depth in the kernel space rather than user space. The Linux kernel's implementation of KMS enables native resolution in the framebuffer and allows for instant console (tty) switching. KMS also enables newer technologies (such as DRI2) which will help reduce artifacts and increase 3D performance, even kernel space power-saving.

https://wiki.recompile.se/wiki/Mandos - a system for allowing servers with encrypted root file systems to reboot unattended and/or remotely. http://www.recompile.se/mandos/man/intro.8mandos







https://en.wikipedia.org/wiki/Device_tree - also written devicetree, is a data structure describing the hardware components of a particular computer so that the operating system's kernel can use and manage those components, including the CPU or CPUs, the memory, the buses and the peripherals.The device tree was derived from SPARC-based workstations and servers via the Open Firmware project. The current Devicetree specification is targeted at smaller systems, but is still used with some server-class systems (for instance, those described by the Power Architecture Platform Reference). Personal computers with the x86 architecture generally do not use device trees, relying instead on various auto configuration protocols to discover hardware. Systems which use device trees usually pass a static device tree (perhaps stored in ROM) to the operating system, but can also generate a device tree in the early stages of booting. As an example, Das U-Boot and kexec can pass a device tree when launching a new operating system. On systems with a boot loader that does not support device trees, a static device tree may be installed along with the operating system; the Linux kernel supports this approach. The Devicetree specification is currently managed by a community named devicetree.org, which is associated with, among others, Linaro and Arm.

init

init (short for initialization) is a daemon process that is the direct or indirect ancestor of all other processes. It automatically adopts all orphaned processes. Init is the first process started during booting, and is typically assigned PID numb