Yori: Guide to Yori

Yori | malsmith home

Yori aims to be a natural and familar shell for those used to operating with CMD. It does not attempt to be fully compatible, because doing so would impact its ability to make improvements. Rather, it aims to be a natural evolution.

  1. Notable differences from CMD
    1. Command wait semantics
    2. Text encoding is UTF-8
    3. VT100/ANSI colors
    4. \\?\ file prefixes
    5. Enhanced file matching
    6. Aliases
  2. Keyboard input
    1. Command history
    2. Tab completion
    3. Completion scripts
  3. Mouse input
  4. Environment variables
    1. YORIAUTORESTART
    2. YORIBACKGROUND
    3. YORICDPATH
    4. YORICOLORAPPEND
    5. YORICOLORMETADATA
    6. YORICOLORPREPEND
    7. YORICOLORREPLACE
    8. YORICOMPLETELISTALL
    9. YORICOMPLETEPATH
    10. YORICOMPLETEWITHTRAILINGSLASH
    11. YORIHISTFILE
    12. YORIHISTSIZE
    13. YORIMOUSEOVER
    14. YORIPRECMD
    15. YORIPOSTCMD
    16. YORIPROMPT
    17. YORIQUICKEDIT
    18. YORIQUICKEDITBREAKCHARS
    19. YORISUGGESTIONDELAY
    20. YORISUGGESTIONMINCHARS
    21. YORITITLE
  5. Using color
    1. Color by escapes
    2. Highlighting files with color
    3. Selecting color for metadata elements
  6. Using backquotes
    1. Escaping backquotes
  7. Modules
  8. Scripting
    1. Subroutine semantics
    2. CMD script semantics
    3. AutoInit scripts
  9. Updating

Notable differences from CMD

Command wait semantics

CMD.EXE will wait for console processes to complete, but will return immediately when GUI processes are launched. Yori uses "&" at the end of commands to indicate whether to return to the shell. "&!" will return to the shell and send the output of the command to an in memory buffer rather than the console; these buffers can be displayed with the job command.

Text encoding is UTF-8

CMD.EXE typically uses ASCII 8-bit characters. Yori outputs in UTF-8, which provides compatibility for the first 127 characters while allowing for far more characters to be expressed. Where ASCII or other encodings are needed, output can be piped through iconv.exe to convert to other encodings.

VT100/ANSI colors

ANSI escapes have traditionally not been provided in Windows NT and its derivatives. Newer versions of Windows 10 provide this capability, but CMD does not enable it. Yori enables this capability for applications, and provides color VT100 support in all of its tools, including prompt and echo.

\\?\ file prefixes

In Windows, argument parsing is performed by the program receiving the string. Programs included with Yori resolve all relative paths to absolute paths with a \\?\ prefix. This allows paths to exceed MAX_PATH, allows files whose names end in a period or a space. It also means that Yori programs treat device names as devices only if a path is not explicitly specified; "CON" is a device, "C:\CON" is not.

Enhanced file matching

Yori programs have several extensions to regular file matching operators. In addition to "*" and "?", Yori supports operators such as "[]" to allow a match of any specified character, or "{}" to allow a match of any specified string in a set. For example, "File[AB]" will match FileA and FileB but not FileC. "File.{ext1,ext2}" will match File.ext1 and File.ext2. Files will be matched with "/" or "\" slashes, and file:/// prefixes are stripped to access the underlying file.

Special directories are also expanded to their system defined value:

~Home directory, as set in %HOMEDRIVE%%HOMEPATH%
~AppdataUser's AppData\Roaming directory
~AppdirDirectory containing the Yori executable
~CommonappdataThe system application data directory
~CommondesktopThe system desktop, shared across all users
~CommondocumentsSystem documents, shared across all users
~CommonprogramsSystem's Start Menu\Programs directory
~CommonstartSystem's Start Menu directory
~DesktopUser's desktop
~DocumentsUser's documents directory
~DownloadsUser's downloads directory
~LocalAppDataUser's AppData\Local directory
~ProgramsUser's Start Menu\Programs directory
~ProgramfilesSystem's Program Files directory
~Programfiles32System's Program Files (x86) directory or Program Files in 32 bit environments
~Programfiles64System's Program Files directory in 64 bit environments
~StartUser's Start Menu directory
~StartupUser's Start Menu\Programs\Startup directory
~SystemSystem's Windows\System32 directory
~WindowsSystem's Windows directory

Aliases

CMD.EXE supports macros via the DOSKEY command. These are expanded by the console, not by CMD, and only if the macro is at the beginning of the line. Yori provides its own alias support which will expand commands regardless of how they appear in the command line, or in scripts. These aliases can be defined by the alias command. Unlike DOSKEY aliases, Yori aliases indicate arguments surrounded by $, as in "$1$" rather than "$1".

Aliases defined for CMD via DOSKEY will be imported and converted for use by Yori when it starts.

Keyboard input

Most keyboard input is modelled on the same keyboard navigation as CMD. Standard left and right arrow, home, end, insert, delete keys behave as expected. Ctrl+Left and Ctrl+Right navigate along command arguments. Ctrl+A moves to the beginning of the line as is common on Linux, and Ctrl+E moves to the end. Ctrl+L can be used to clear the screen. Shift+Left and Shift+Right can be used to select text to allow for overwrite or delete.

Unlike CMD, Yori waits for all processes, including GUI processes. To end a wait and return to the shell, use Ctrl+B.

Command history

The up arrow key moves to the previous command. Unlike CMD, the down arrow will not move to the "next" command; command history is unidirectional, with the most recent command at the bottom, and up moving to progressively less recent commands. Ctrl+Up will take a newly entered characters and find previously entered commands with the same starting characters. Ctrl+Del will delete a command from history and move to the previous command.

Tab completion

Tab will attempt to complete a command or argument. By default, tab in the first argument attempts to match an executable program, and tab for later arguments attempts to match file and directory matches. Ctrl+Tab will perform matches by completing the full path rather than just a file name.

By default, Yori implements tab completion in a CMD-like fashion. It can optionally add trailing backslashes, as PowerShell does, by setting YORICOMPLETEWITHTRAILINGSLASH=1. It can optionally prompt for all available matches and wait for the user to specify another character, as bash does, by setting YORICOMPLETELISTALL=1.

Completion scripts

Custom completion rules can be defined for specific programs in %YORICOMPLETEPATH%. These scripts can be Yori scripts or any other executable. The first argument to a completion script is the argument number being completed; the second argument is the argument string that has been entered so far requiring completion. A completion script can output:

/commandsMatches executables and builtin commands
/directoriesMatches directories only (not files)
/executablesMatches executables only
/filesMatches files and directories
/filesonlyMatches files (not directories)
/insensitivelist <strings>Matches against an explicitly specified list, case insensitively
/sensitivelist <strings>Matches against an explicitly specified list, case sensitively

Mouse input

Yori is capable of handling mouse input in one of three ways:

With YORIQUICKEDIT or Windows Quickedit disabled, when at the prompt, Ctrl+Click can be used to select text that is displayed on the console to insert into the current command. This text is highlighted. To disable this highlight, set YORIMOUSEOVER to 0.

Yori's mouse selection is similar to QuickEdit but has some differences:

Environment variables

YORIAUTORESTART

When set to 1, Yori will periodically save the state of the shell and the window so that if Yori crashes or Windows Update restarts the system the state will be automatically restored. The default is off. Optionally the 1 can be followed by a comma and the number of lines of screen buffer to record. Saving a smaller amount of screen buffer can improve performance of the save operation.

YORIBACKGROUND

When set to 1, Yori will attempt to use background colors on Nano server. Nano Server 2016 has a bug that prevents background colors from working correctly, so setting this indicates a patched kernel with the bug fixed. Background colors are displayed on non-Nano servers regardless of this value.

YORICDPATH

Specifies a semicolon delimited list of paths to check for matches when a relative path is given to chdir or cd. Typically one element in this list should be "." for the current directory, but frequently used directories can also be consulted for fast access.

YORICOLORAPPEND

Specifies the critieria to apply when highlighting files with color. Entries in this list are processed in order after any system defaults. For more information, see highlighting files with color.

YORICOLORMETADATA

Specifies the critieria to apply to file metadata and UI elements. Entries in this list are processed in order before any system defaults. For more information, see selecting color for metadata elements.

YORICOLORPREPEND

Specifies the critieria to apply when highlighting files with color. Entries in this list are processed in order and before any system defaults. For more information, see highlighting files with color.

YORICOLORREPLACE

Specifies the critieria to apply when highlighting files with color. Entries in this list replace any system defaults. For more information, see highlighting files with color.

YORICOMPLETELISTALL

When set to 1, if tab completion could indicate multiple options, each option is displayed and the user is prompted to specify additional characters to indicate a choice. By default, tab completion cycles between available matches.

YORICOMPLETEPATH

Specifies a semicolon delimited list of paths to check for completion scipts or programs. When a tab completion is invoked for a program, the completion script, if present, determines which options are available for the tab completion. If no completion script is found, files and directories are used.

YORICOMPLETEWITHTRAILINGSLASH

When set to 1, when tab completion is completing a directory, it will add a trailing backslash. By default, no trailing backslash is included.

YORIHISTFILE

If specified, provides a file to save command history to when the Yori process exits, and to load history from when the process is started.

YORIHISTSIZE

If specified, provides the number of commands that should be retained as command history. The current default, as of this writing, is 250.

YORIMOUSEOVER

If specified, and set to zero, disables the default behavior of highlighting text which can be inserted into the current command with Ctrl+Click. Note that disabling the highlight does not disable Ctrl+click behavior.

YORIPRECMD

If specified, contains a command to execute after the user has entered a command to execute but before the user entered command is executed.

YORIPOSTCMD

If specified, contains a command to execute after the user entered command has finished executing but before the prompt command is executed.

YORIPROMPT

Defines the prompt value to use for Yori. This is similar to CMD's PROMPT variable, although variables are surrounded by $, as in "$P$" rather than CMD's "$P". YORIPROMPT can contain environment variables to expand, or backquotes to execute each time the prompt is displayed. Defined variables for the prompt are:

$A$&
$B$|
$C$(
$E$Escape character. Used to initiate VT100 sequences.
$F$)
$G$>
$G_OR_ADMIN_G$> for a non-administrative prompt, » for an administrative prompt
$L$<
$P$Current directory
$Q$=
$S$Space
$_$New line

YORIQUICKEDIT

If set to 1, Yori will use its internal mouse support when entering text at the prompt, and the console quickedit when running applications. If not set, the current console mode (either quickedit or not) will be used constantly. For more information, see mouse input.

YORIQUICKEDITBREAKCHARS

When double clicking on the console window to select items, Yori will end the selection if any character in the YORIQUICKEDITBREAKCHARS variable is encountered. By default, this includes space, apostrophe, greater than, less than, and vertical bar symbols. This variable can include either characters or a comma seperated list of character codes. For more information, see mouse input.

YORISUGGESTIONDELAY

Specifies the amount of time to wait, in milliseconds, before suggesting a completion for the entered command. The default, as of this writing, is 400 milliseconds. If this value is set to zero, suggestions are disabled.

YORISUGGESTIONMINCHARS

Specifies the number of characters that must be entered in a path before suggestions should be made. Because an empty string could match anything, this value is used to only suggest things once enough data is present to make the suggestions meaningful. The default is 2 characters.

YORITITLE

This variable behaves the same as YORIPROMPT, including expanding environment variables and backquotes, and sets the title of the window after each command.

Using color

Color by escapes

Yori supports VT100 escapes to provide color. This means that if it encounters a raw VT100 string, such as "<ESC>[31m" it will apply the requested color rather than displaying the literal text. A raw VT100 string can be entered on the command line by holding down Alt, then pressing 2, then 7, then releasing Alt. This inserts character 27, which is the VT100 escape character. Subsequent letters can be typed normally. When using color with a prompt, $E$ can be used to generate the ESC character, so manually entering the character is not needed. Futher, the raw VT100 codes can be generated by using the supplied COLOR command. Typically the COLOR command is used in backquotes so as to generate the VT100 escape sequence and supply it to some other command, such as ECHO. Without doing this, the current color will be changed but no text will be displayed using the changed color.

Sequences to generate foreground colors are:

COLOR black<ESC>[30m
COLOR red<ESC>[31m
COLOR green<ESC>[32m
COLOR brown<ESC>[33m
COLOR blue<ESC>[34m
COLOR magenta<ESC>[35m
COLOR cyan<ESC>[36m
COLOR gray<ESC>[37m
COLOR darkgray<ESC>[30;1m
COLOR lightred<ESC>[31;1m
COLOR lightgreen<ESC>[32;1m
COLOR yellow<ESC>[33;1m
COLOR lightblue<ESC>[34;1m
COLOR lightmagenta<ESC>[35;1m
COLOR lightcyan<ESC>[36;1m
COLOR white<ESC>[37;1m

Sequences to generate background colors are:

COLOR bg_black<ESC>[40m
COLOR bg_red<ESC>[41m
COLOR bg_green<ESC>[42m
COLOR bg_brown<ESC>[43m
COLOR bg_blue<ESC>[44m
COLOR bg_magenta<ESC>[45m
COLOR bg_cyan<ESC>[46m
COLOR bg_gray<ESC>[47m
COLOR bg_darkgray<ESC>[100m
COLOR bg_lightred<ESC>[101m
COLOR bg_lightgreen<ESC>[102m
COLOR bg_yellow<ESC>[103m
COLOR bg_lightblue<ESC>[104m
COLOR bg_lightmagenta<ESC>[105m
COLOR bg_lightcyan<ESC>[106m
COLOR bg_white<ESC>[107m

Both in raw VT100 form and from the COLOR command, multiple colors can be combined. To set yellow text on a blue background in VT100, use "<ESC>[44;33;1m". To set yellow text on a blue background from the COLOR command, use "color bg_blue+yellow".

Highlighting files with color

Some tools within Yori support displaying files in color based on criteria about the file. These tools include dir, du, and sdir. Color coding for files is defined via three environment variables, processed in order:

  1. YORICOLORPREPEND, processed first
  2. YORICOLORREPLACE, if defined; otherwise, built in defaults apply
  3. YORICOLORAPPEND, processed last

Each variable contains a semicolon delimited list of rules. Each rule takes the form of:

[file attribute][operator][criteria],[color]

Valid operators are:

Valid attributes and corresponding operators are:

The default set of file color attributes is:

Selecting color for metadata elements

Color coding for metadata attributes is defined via YORICOLORMETADATA. This variable also contains a semicolon delimited list of rules. Each rule takes the form of:

[file attribute],[color]

Common metadata attributes and their current defaults are:

file sizefsyellow
mouse overmounderline+lightblue
number filesnflightgreen

Sdir has additional options for its metadata elements. See sdir -? -metacolor for more information.

Using backquotes

Backquotes work by firstly parsing a command line to find matching instances of the "`" character, then executing the command contained between the match and substituting the output of the command into the command line. This is very useful to execute one command to drive the action of another command.

Escaping backquotes

Frequently when creating a command line a choice must be made about when backquote expressions should be executed. By default, these are executed before performing the command line. However, often it is useful to pass the backquote expression to another command, such as for or set. Doing this requires escaping the backquotes, so they are not executed initially but are instead preserved as a string to the next command. As in CMD, Yori uses the "^" character to indicate the following character should be handled literally. For example:

    SET YORIPROMPT=`whoami` $P$$G$
    

Will evaluate "whoami" into a string, then place that string hardcoded into the prompt. However, the prompt can also evaluate the expression every time the prompt is displayed, even if the state changes. Doing this requires escaping to ensure that the SET command receives the backquoted string:

    SET YORIPROMPT=^`whoami^` $P$$G$
    

The same approach applies to environment variables. Consider the following:

    SET YORIPROMPT=%USERNAME% $P$$G$
    SET YORIPROMPT=^%USERNAME^% $P$$G$
    

Modules

Many commands in any shell need to be evaluated within the shell process itself. Some state, such as the current directory or environment, affects the shell process and these commands are only meaningful as in-process commands. Many shells, including CMD.EXE, only support internal commands coded into the shell. Yori allows in-process commands to be added externally and loaded in the form of DLLs with a .COM extension. Any .COM file, when executed, is checked to see if is it a DLL to execute in process, or if it is a regular executable.

Executables in the PATH used in preference to statically provided builtin commands. This is the oppose of CMD's behavior, and is provided so that internal commands can be superseded with enhanced versions in any environment. Every internal command supported by Yori is generated as a DLL as part of the build process, and can be altered or extended as needed.

Modules depend on being able to manipulate process state. In some cases, this can be done by calling the Windows API directly. To interact with functionality provided by the shell, YORI.EXE exports functions which modules can call.

Scripting

Yori does not execute CMD scripts. CMD scripts are executed by CMD. Yori executes its own scripts, which end in a .YS1 extension. When a script is executed, Yori provides extra commands that are not available or meaningful from an interactive command line. These include CALL, GOTO, INCLUDE, RETURN, and SHIFT. Help about these commands is available from "YS /?".

Unlike CMD, a Yori script can invoke another script without terminating execution of the first script (CALL is not required for this.) Environment variables are evaluated when each line is executed (which CMD optionally does via SETLOCAL ENABLEDELAYEDEXPANSION.) Arguments in scripts are referred with training and leading %, as in %1%, whereas CMD uses %1 only. %*% matches all arguments.

Subroutine semantics

Labels within scripts follow the same syntax as CMD, being ":Label" at the beginning of a line. GOTO works similarly to CMD. However, Yori introduces CALL to execute a subroutine, which returns to the call site via the RETURN command. When calling a subroutine, %1% et al refer to arguments passed to the subroutine. Changes within the subroutine are not returned to the calling function by default. The RETURN command specifies which variables should have their changes made to global scope. Consider this simple example:

    CALL subroutine Arg1
    ECHO LOCALVARIABLE=%LOCALVARIABLE% GLOBALVARIABLE=%GLOBALVARIABLE%
    GOTO :eof

    :subroutine
    SET LOCALVARIABLE=%1%
    SET GLOBALVARIABLE=%1%
    RETURN 0 GLOBALVARIABLE
    

In this example, only GLOBALVARIABLE will be set in the main scope, since it was the only value explicitly returned from the subroutine.

AutoInit scripts

Because Yori's configuration is driven through environment variables, scripts can be used to customize a Yori environment, similar to profile scripts on other shells. Note in particular that Yori has no configuration files; configuration is driven through scripts, as is common for many shells. Although these scripts are typically .YS1 scripts, they can also be .CMD scripts or any other program. Yori searches for scripts to automatically execute in the following locations:

  1. Scripts under the ~AppDir\YoriInit.d directory. ie., system-wide files placed in the YoriInit.d subdirectory underneath the directory containing Yori.exe.
  2. Any script called ~AppDir\YoriInit*. ie., system-wide files placed in the same directory as Yori.exe whose name begins with YoriInit.
  3. Scripts under the ~\YoriInit.d directory. ie., per-user files placed in the YoriInit.d subdirectory underneath %HOMEDRIVE%%HOMEPATH%.
  4. Any script called ~\YoriInit*. ie., per-user files placed in %HOMEDRIVE%%HOMEPATH% whose name begins with YoriInit.

Note that as with CMD, it is often useful to have per-shortcut settings. This can be accomplished by adding "/k path_to_script.ys1" to the arguments in the shortcut.

CMD script semantics

Yori executes CMD scripts with CMD. However, to provide compatibility with CMD's behavior when executing CMD scripts, it will try to apply changes made by the script to the environment and aliases back to the Yori process. This means that a CMD script containing "SET FOO=BAR" should result in %FOO% being defined in Yori when the script has finished executing.

Updating

Yori and its tools can be updated via the Yori Package Manager, or ypm. In the typical case, running "ypm -u" will update your system to the latest version. Note this requires write access to the Yori binaries, which will be replaced if they are out of date.

Ypm can also be used to install and uninstall packages with a fully qualified file name or URL. It can also search for packages in a list of sources, either local or remote. By default, ypm will use www.malsmith.net as a source. This can be modified by creating a [Sources] section in packages.ini, and adding Source<n>=<URL> lines where n is a monotonically increasing number.

Ypm can also mirror packages, re-routing requests destined to one source to another source. To do this, add a [Mirrors] section in packages.ini with each line containing a value to substitute with another value. The '%' character can be used to indicate an '=' character, which is otherwise inexpressible because it delimits the value to replace with the new value to use. For example, to remap malsmith.net packages to a local location, use:

    [Mirrors]
    http://www.malsmith.net/download/?obj%yori/latest-stable/=c:\local\packages\