A Practical Guide to Linux Commands, Editors, and Shell Programming

The Most Useful Tutorial and Reference, with Hundreds of High-Quality Examples for Every Popular Linux Distribution “First Sobell taught people how to use Linux . . . now he teaches you the power of Linux. A must-have book for anyone who wants to take Linux to the next level.” –Jon “maddog” Hall, Executive Director, Linux International Discover the Power of Linux—Covers macOS, too! Learn from hundreds of realistic, high-quality examples, and become a true command-line guru Covers MariaDB, DNF, and Python 3 300+ page reference section covers 102 utilities, including macOS commands For use with all popular versions of Linux, including Ubuntu,™ Fedora,™ openSUSE,™ Red Hat,® Debian, Mageia, Mint, Arch, CentOS, and macOS   Linux is today’s dominant Internet server platform. System administrators and Web developers need deep Linux fluency, including expert knowledge of shells and the command line. This is the only guide with everything you need to achieve that level of Linux mastery. Renowned Linux expert Mark Sobell has brought together comprehensive, insightful guidance on the tools sysadmins, developers, and power users need most, and has created an outstanding day-to-day reference, updated with assistance from new coauthor Matthew Helmke.   This title is 100 percent distribution and release agnostic. Packed with hundreds of high-quality, realistic examples, it presents Linux from the ground up: the clearest explanations and most useful information about everything from filesystems to shells, editors to utilities, and programming tools to regular expressions.   Use a Mac? You’ll find coverage of the macOS command line, including macOS-only tools and utilities that other Linux/UNIX titles ignore.   A Practical Guide to Linux® Commands, Editors, and Shell Programming, Fourth Edition, is the only guide to deliver A MariaDB chapter to get you started with this ubiquitous relational database management system (RDBMS) A masterful introduction to Python for system administrators and power users In-depth coverage of the bash and tcsh shells, including a complete discussion of environment, inheritance, and process locality, plus coverage of basic and advanced shell programming Practical explanations of core utilities, from aspell to xargs, including printf and sshfs/curlftpfs, PLUS macOS–specific utilities from ditto to SetFile Expert guidance on automating remote backups using rsync Dozens of system security tips, including step-by-step walkthroughs of implementing secure communications using ssh and scp Tips and tricks for customizing the shell, including step values, sequence expressions, the eval builtin, and implicit command-line continuation High-productivity editing techniques using vim and emacs A comprehensive, 300-plus-page command reference section covering 102 utilities, including find, grep, sort, and tar Instructions for updating systems using apt-get and dnf And much more, including coverage of BitTorrent, gawk, sed, find, sort, bzip2, and regular expressions

Table of Contents:
Preface xxxv   Chapter 1: Welcome to Linux and macOS 1 The History of UNIX and GNU–Linux 3 What Is So Good About Linux? 6 Overview of Linux 11 Additional Features of Linux 16 Chapter Summary 18 Exercises 18   Part I: The Linux and macOS Operating Systems 21   Chapter 2: Getting Started 23 Conventions Used in This Book 24 Logging In from a Terminal (Emulator) 26 Working from the Command Line 28 su/sudo: Curbing Your Power (root Privileges) 32 Where to Find Documentation 33 More About Logging In and Passwords 42 Chapter Summary 46 Exercises 47 Advanced Exercises 48   Chapter 3: The Utilities 49 Special Characters 50 Basic Utilities 51 Working with Files 53 | (Pipeline): Communicates Between Processes 60 Four More Utilities 61 Compressing and Archiving Files 64 Locating Utilities 69 Displaying User and System Information 71 Communicating with Other Users 75 Email 77 Chapter Summary 77 Exercises 80 Advanced Exercises 81   Chapter 4: The Filesystem 83 The Hierarchical Filesystem 84 Directory Files and Ordinary Files 85 Pathnames 90 Working with Directories 92 Access Permissions 100 ACLs: Access Control Lists 106 Links 112 Chapter Summary 122 Exercises 124 Advanced Exercises 126   Chapter 5: The Shell 127 Special Characters 128 Ordinary Files and Directory Files 129 The Command Line 130 Standard Input and Standard Output 137 Running a Command in the Background 150 Filename Generation/Pathname Expansion 152 Builtins 157 Chapter Summary 158 Exercises 159 Advanced Exercises 160   Part II: The Editors 163   Chapter 6: The vim Editor 165 History 166 Tutorial: Using vim to Create and Edit a File 167 Introduction to vim Features 175 Command Mode: Moving the Cursor 181 Input Mode 185 Command Mode: Deleting and Changing Text 186 Searching and Substituting 190 Miscellaneous Commands 197 Copying, Moving, and Deleting Text 197 Reading and Writing Files 200 Setting Parameters 201 Advanced Editing Techniques 206 Units of Measure 210 Chapter Summary 213 Exercises 218 Advanced Exercises 219   Chapter 7: The emacs Editor 221 History 222 Tutorial: Getting Started with emacs 224 Basic Editing Commands 231 Online Help 238 Advanced Editing 240 Major Modes: Language-Sensitive Editing 255 Customizing emacs 265 More Information 270 Chapter Summary 270 Exercises 279 Advanced Exercises 280   Part III: The Shells 283   Chapter 8: The Bourne Again Shell (bash) 285 Background 286 Startup Files 288 Commands That Are Symbols 291 Redirecting Standard Error 292 Writing and Executing a Simple Shell Script 294 Control Operators: Separate and Group Commands 299 Job Control 304 Manipulating the Directory Stack 307 Parameters and Variables 310 Special Characters 325 Locale 326 Time 330 Processes 333 History 336 Aliases 352 Functions 356 Controlling bash: Features and Options 359 Processing the Command Line 364 Chapter Summary 374 Exercises 376 Advanced Exercises 378   Chapter 9: The TC Shell (tcsh) 379 Shell Scripts 380 Entering and Leaving the TC Shell 381 Features Common to the Bourne Again and TC Shells 383 Redirecting Standard Error 389 Working with the Command Line 390 Variables 396 Control Structures 408 Builtins 418 Chapter Summary 422 Exercises 423 Advanced Exercises 425   Part IV: Programming Tools 427   Chapter 10: Programming the Bourne Again Shell (bash) 429 Control Structures 430 File Descriptors 464 Parameters 470 Variables 479 Builtin Commands 489 Expressions 505 Implicit Command-Line Continuation 512 Shell Programs 513 Chapter Summary 523 Exercises 525 Advanced Exercises 527   Chapter 11: The Perl Scripting Language 529 Introduction to Perl 530 Variables 538 Control Structures 545 Working with Files 554 Sort 558 Subroutines 559 Regular Expressions 562 CPAN Modules 568 Examples 570 Chapter Summary 574 Exercises 574 Advanced Exercises 575   Chapter 12: The Python Programming Language 577 Introduction 578 Scalar Variables, Lists, and Dictionaries 582 Control Structures 588 Reading from and Writing to Files 593 Regular Expressions 597 Defining a Function 598 Using Libraries 599 Lambda Functions 603 List Comprehensions 604 Chapter Summary 605 Exercises 606 Advanced Exercises 606   Chapter 13: The MariaDB SQL Database Management System 609 History 610 Notes 611 Installing a MariaDB Server and Client 614 Client Options 615 Setting Up MariaDB 616 Creating a Database 618 Adding a User 619 Examples 620 Chapter Summary 633 Exercises 633   Chapter 14: The AWK Pattern Processing Language 635 Syntax 636 Arguments 636 Options 637 Notes 638 Language Basics 638 Examples 645 Advanced gawk Programming 662 Chapter Summary 667 Exercises 668 Advanced Exercises 668   Chapter 15: The sed Editor 669 Syntax 670 Arguments 670 Options 670 Editor Basics 671 Examples 674 Chapter Summary 685 Exercises 685   Part V: Secure Network Utilities 687   Chapter 16: The rsync Secure Copy Utility 689 Syntax 690 Arguments 690 Options 691 Examples 693 Chapter Summary 700 Exercises 701   Chapter 17: The OpenSSH Secure Communication Utilities 703 Introduction to OpenSSH 704 Running the ssh, scp, and sftp OpenSSH Clients 706 Setting Up an OpenSSH Server (sshd) 717 Troubleshooting 724 Tunneling/Port Forwarding 724 Chapter Summary 727 Exercises 728 Advanced Exercises 728   Part VI: Command Reference 729   Utilities That Display and Manipulate Files 731 Network Utilities 732 Utilities That Display and Alter Status 733 Utilities That Are Programming Tools 734 Miscellaneous Utilities 734 Standard Multiplicative Suffixes 735 Common Options 736 The sample Utility 736 sample: Brief description of what the utility does (macOS) 737 aspell: Checks a file for spelling errors 739 at: Executes commands at a specified time 743 busybox: Implements many standard utilities 747 bzip2: Compresses or decompresses files 750 cal: Displays a calendar 752 cat: Joins and displays files 753 cd: Changes to another working directory 755 chgrp: Changes the group associated with a file 757 chmod: Changes the access mode (permissions) of a file 759 chown: Changes the owner of a file and/or the group the file is associated with 764 cmp: Compares two files 766 comm: Compares sorted files 768 configure: Configures source code automatically 770 cp: Copies files 772 cpio: Creates an archive, restores files from an archive, or copies a directory hierarchy 776 crontab: Maintains crontab files 781 cut: Selects characters or fields from input lines 784 date: Displays or sets the system time and date 787 dd: Converts and copies a file 790 df: Displays disk space usage 793 diff: Displays the differences between two text files 795 diskutil: Checks, modifies, and repairs local volumes (macOS) 800 ditto: Copies files and creates and unpacks archives (macOS) 803 dmesg: Displays kernel messages 805 dscl: Displays and manages Directory Service information (macOS) 806 du: Displays information on disk usage by directory hierarchy and/or file 809 echo: Displays a message 812 expand/unexpand: Converts TABs to SPACEs and SPACEs to TABs 814 expr: Evaluates an expression 816 file: Displays the classification of a file 820 find: Finds files based on criteria 822 finger: Displays information about users 828 fmt: Formats text very simply 831 fsck: Checks and repairs a filesystem 833 ftp: Transfers files over a network 838 gawk: Searches for and processes patterns in a file 845 gcc: Compiles C and C++ programs 846 GetFileInfo: Displays file attributes (macOS) 851 grep: Searches for a pattern in files 853 gzip: Compresses or decompresses files 858 head: Displays the beginning of a file 861 join: Joins lines from two files based on a common field 863 kill: Terminates a process by PID 866 killall: Terminates a process by name 868 launchctl: Controls the launchd daemon (macOS) 870 less: Displays text files, one screen at a time 873 ln: Makes a link to a file 878 lpr: Sends files to printers 881 ls: Displays information about one or more files 884 make: Keeps a set of programs current 892 man: Displays documentation for utilities 898 mc: Manages files in a textual environment (aka Midnight Commander) 902 mkdir: Creates a directory 909 mkfs: Creates a filesystem on a device 911 mv: Renames or moves a file 914 nice: Changes the priority of a command 916 nl: Numbers lines from a file 918 nohup: Runs a command that keeps running after you log out 920 od: Dumps the contents of a file 921 open: Opens files, directories, and URLs (macOS) 926 otool: Displays object, library, and executable files O 928 paste: Joins corresponding lines from files 930 pax: Creates an archive, restores files from an archive, or copies a directory hierarchy 932 plutil: Manipulates property list files (macOS) 938 pr: Paginates files for printing 940 printf: Formats string and numeric data 942 ps: Displays process status 946 renice: Changes the priority of a process 951 rm: Removes a file (deletes a link) 953 rmdir: Removes directories 955 rsync: Securely copies files and directory hierarchies over a network 956 scp: Securely copies one or more files to or from a remote system 957 screen: Manages several textual windows 958 sed: Edits a file noninteractively 964 SetFile: Sets file attributes (macOS) 965 sleep: Creates a process that sleeps for a specified interval 967 sort: Sorts and/or merges files 969 split: Divides a file into sections 978 ssh: Securely runs a program or opens a shell on a remote system 980 sshfs/curlftpfs: Mounts a directory on an OpenSSH or FTP server as a local directory 981 stat: Displays information about files 984 strings: Displays strings of printable characters from files 986 stty: Displays or sets terminal parameters 987 sysctl: Displays and alters kernel variables at runtime 991 tail: Displays the last part (tail) of a file 992 tar: Stores or retrieves files to/from an archive file 995 tee: Copies standard input to standard output and one or more files 1000 telnet: Connects to a remote computer over a network 1001 test: Evaluates an expression 1005 top: Dynamically displays process status 1008 touch: Creates a file or changes a file’s access and/or modification time 1012 tr: Replaces specified characters 1014 tty: Displays the terminal pathname 1017 tune2fs: Changes parameters on an ext2, ext3, or ext4 filesystem 1018 umask: Specifies the file-creation permissions mask 1021 uniq: Displays unique lines from a file 1023 w: Displays information about local system users 1025 wc: Displays the number of lines, words, and bytes in one or more files 1027 which: Shows where in PATH a utility is located 1028 who: Displays information about logged-in users 1030 xargs: Converts standard input to command lines 1032   Part VII: Appendixes 1035   Appendix A: Regular Expressions 1037 Characters 1038 Delimiters 1038 Simple Strings 1038 Special Characters 1038 Rules 1041 Bracketing Expressions 1042 The Replacement String 1042 Extended Regular Expressions 1043 Appendix Summary 1045   Appendix B: Help 1047 Solving a Problem 1048 Finding Linux and macOS Related Information 1049 Specifying a Terminal 1050   Appendix C: Keeping the System Up-to-Date 1053 Using dnf 1054 Using apt-get 1060 BitTorrent 1064   Appendix D: macOS Notes 1067 Open Directory 1068 Filesystems 1069 Extended Attributes 1070 Activating the Terminal META Key 1076 Startup Files 1076 Remote Logins 1076 Many Utilities Do Not Respect Apple Human Interface Guidelines 1076 Installing Xcode and MacPorts 1077 macOS Implementation of Linux Features 1078   Glossary 1081 File Tree Index 1135 Utility Index 1137 Main Index 1141