Yori: Guide to Yori

Yori | malsmith home

Yori aims to be a natural and familar shell for those used to operating with CMD. It does not attempt to be fully compatible, because doing so would impact its ability to make improvements. Rather, it aims to be a natural evolution.

  1. Notable differences from CMD
    1. Command wait semantics
    2. Text encoding is UTF-8
    3. VT100/ANSI colors
    4. \\?\ file prefixes
    5. Enhanced file matching
    6. Aliases
  2. Keyboard input
    1. Command history
    2. Tab completion
    3. Completion scripts
  3. Mouse input
  4. Environment variables
    1. YORIAUTORESTART
    2. YORICOMPLETEPATH
    3. YORIHISTFILE
    4. YORIHISTSIZE
    5. YORIPROMPT
    6. YORIQUICKEDIT
    7. YORIQUICKEDITBREAKCHARS
    8. YORISUGGESTIONDELAY
    9. YORISUGGESTIONMINCHARS
    10. YORITITLE
  5. Using color
  6. Using backquotes
    1. Escaping backquotes
  7. Modules
  8. Scripting
    1. Subroutine semantics
    2. CMD script semantics
  9. Updating

Notable differences from CMD

Command wait semantics

CMD.EXE will wait for console processes to complete, but will return immediately when GUI processes are launched. Yori uses "&" at the end of commands to indicate whether to return to the shell. "&!" will return to the shell and send the output of the command to an in memory buffer rather than the console; these buffers can be displayed with the job command.

Text encoding is UTF-8

CMD.EXE typically uses ASCII 8-bit characters. Yori outputs in UTF-8, which provides compatibility for the first 127 characters while allowing for far more characters to be expressed. Where ASCII or other encodings are needed, output can be piped through iconv.exe to convert to other encodings.

VT100/ANSI colors

ANSI escapes have traditionally not been provided in Windows NT and its derivatives. Newer versions of Windows 10 provide this capability, but CMD does not enable it. Yori enables this capability for applications, and provides color VT100 support in all of its tools, including prompt and echo.

\\?\ file prefixes

In Windows, argument parsing is performed by the program receiving the string. Programs included with Yori resolve all relative paths to absolute paths with a \\?\ prefix. This allows paths to exceed MAX_PATH, allows files whose names end in a period or a space. It also means that Yori programs treat device names as devices only if a path is not explicitly specified; "CON" is a device, "C:\CON" is not.

Enhanced file matching

Yori programs have several extensions to regular file matching operators. In addition to "*" and "?", Yori supports operators such as "[]" to allow a match of any specified character, or "{}" to allow a match of any specified string in a set. For example, "File[AB]" will match FileA and FileB but not FileC. "File.{ext1,ext2}" will match File.ext1 and File.ext2. Files will be matched with "/" or "\" slashes, and file:/// prefixes are stripped to access the underlying file.

Special directories are also expanded to their system defined value:

~Home directory, as set in %HOMEDRIVE%%HOMEPATH%
~AppdataUser's AppData\Roaming directory
~DesktopUser's desktop
~DocumentsUser's documents directory
~DownloadsUser's downloads directory
~LocalAppDataUser's AppData\Local directory
~ProgramsUser's Start Menu\Programs directory
~StartUser's Start Menu directory
~StartupUser's Start Menu\Programs\Startup directory

Aliases

CMD.EXE supports macros via the DOSKEY command. These are expanded by the console, not by CMD, and only if the macro is at the beginning of the line. Yori provides its own alias support which will expand commands regardless of how they appear in the command line, or in scripts. These aliases can be defined by the alias command. Unlike DOSKEY aliases, Yori aliases indicate arguments surrounded by $, as in "$1$" rather than "$1".

Aliases defined for CMD via DOSKEY will be imported and converted for use by Yori when it starts.

Keyboard input

Most keyboard input is modelled on the same keyboard navigation as CMD. Standard left and right arrow, home, end, insert, delete keys behave as expected. Ctrl+Left and Ctrl+Right navigate along command arguments. Ctrl+A moves to the beginning of the line as is common on Linux, and Ctrl+E moves to the end. Ctrl+L can be used to clear the screen. Shift+Left and Shift+Right can be used to select text to allow for overwrite or delete.

Command history

The up arrow key moves to the previous command. Unlike CMD, the down arrow will not move to the "next" command; command history is unidirectional, with the most recent command at the bottom, and up moving to progressively less recent commands. Ctrl+Up will take a newly entered characters and find previously entered commands with the same starting characters. Ctrl+Del will delete a command from history and move to the previous command.

Tab completion

Tab will attempt to complete a command or argument. By default, tab in the first argument attempts to match an executable program, and tab for later arguments attempts to match file and directory matches. Ctrl+Tab will perform matches by completing the full path rather than just a file name.

Completion scripts

Custom completion rules can be defined for specific programs in %YORICOMPLETEPATH%. These scripts can be Yori scripts or any other executable. The first argument to a completion script is the argument number being completed; the second argument is the argument string that has been entered so far requiring completion. A completion script can output:

/commandsMatches executables and builtin commands
/directoriesMatches directories only (not files)
/executablesMatches executables only
/filesMatches files and directories
/filesonlyMatches files (not directories)
/insensitivelist <strings>Matches against an explicitly specified list, case insensitively
/sensitivelist <strings>Matches against an explicitly specified list, case sensitively

Mouse input

Yori is capable of handling mouse input in one of three ways:

Yori's mouse selection is similar to QuickEdit but has some differences:

Environment variables

YORIAUTORESTART

When set to 1, Yori will periodically save the state of the shell and the window so that if Yori crashes or Windows Update restarts the system the state will be automatically restored. The default is off.

YORICOMPLETEPATH

Specifies a semicolon delimited list of paths to check for completion scipts or programs. When a tab completion is invoked for a program, the completion script, if present, determines which options are available for the tab completion. If no completion script is found, files and directories are used.

YORIHISTFILE

If specified, provides a file to save command history to when the Yori process exits, and to load history from when the process is started.

YORIHISTSIZE

If specified, provides the number of commands that should be retained as command history. The current default, as of this writing, is 250.

YORIPROMPT

Defines the prompt value to use for Yori. This is similar to CMD's PROMPT variable, although variables are surrounded by $, as in "$P$" rather than CMD's "$P". YORIPROMPT can contain environment variables to expand, or backquotes to execute each time the prompt is displayed. Defined variables for the prompt are:
$A$&
$B$|
$C$(
$E$Escape character. Used to initiate VT100 sequences.
$F$)
$G$>
$L$<
$P$Current directory
$Q$=
$S$Space
$_$New line

YORIQUICKEDIT

If set to 1, Yori will use its internal mouse support when entering text at the prompt, and the console quickedit when running applications. If not set, the current console mode (either quickedit or not) will be used constantly. For more information, see mouse input.

YORIQUICKEDITBREAKCHARS

When double clicking on the console window to select items, Yori will end the selection if any character in the YORIQUICKEDITBREAKCHARS variable is encountered. By default, this includes space, apostrophe, greater than, less than, and vertical bar symbols. For more information, see mouse input.

YORISUGGESTIONDELAY

Specifies the amount of time to wait, in milliseconds, before suggesting a completion for the entered command. The default, as of this writing, is 400 milliseconds. If this value is set to zero, suggestions are disabled.

YORISUGGESTIONMINCHARS

Specifies the number of characters that must be entered in a path before suggestions should be made. Because an empty string could match anything, this value is used to only suggest things once enough data is present to make the suggestions meaningful. The default is 2 characters.

YORITITLE

This variable behaves the same as YORIPROMPT, including expanding environment variables and backquotes, and sets the title of the window after each command.

Using color

Yori supports VT100 escapes to provide color. This means that if it encounters a raw VT100 string, such as "<ESC>[31m" it will apply the requested color rather than displaying the literal text. A raw VT100 string can be entered on the command line by holding down Alt, then pressing 2, then 7, then releasing Alt. This inserts character 27, which is the VT100 escape character. Subsequent letters can be typed normally. When using color with a prompt, $E$ can be used to generate the ESC character, so manually entering the character is not needed. Futher, the raw VT100 codes can be generated by using the supplied COLOR command. Typically the COLOR command is used in backquotes so as to generate the VT100 escape sequence and supply it to some other command, such as ECHO. Without doing this, the current color will be changed but no text will be displayed using the changed color.

Sequences to generate foreground colors are:

COLOR black<ESC>[30m
COLOR red<ESC>[31m
COLOR green<ESC>[32m
COLOR brown<ESC>[33m
COLOR blue<ESC>[34m
COLOR magenta<ESC>[35m
COLOR cyan<ESC>[36m
COLOR gray<ESC>[37m
COLOR darkgray<ESC>[30;1m
COLOR lightred<ESC>[31;1m
COLOR lightgreen<ESC>[32;1m
COLOR yellow<ESC>[33;1m
COLOR lightblue<ESC>[34;1m
COLOR lightmagenta<ESC>[35;1m
COLOR lightcyan<ESC>[36;1m
COLOR white<ESC>[37;1m

Sequences to generate background colors are:

COLOR bg_black<ESC>[40m
COLOR bg_red<ESC>[41m
COLOR bg_green<ESC>[42m
COLOR bg_brown<ESC>[43m
COLOR bg_blue<ESC>[44m
COLOR bg_magenta<ESC>[45m
COLOR bg_cyan<ESC>[46m
COLOR bg_gray<ESC>[47m
COLOR bg_darkgray<ESC>[100m
COLOR bg_lightred<ESC>[101m
COLOR bg_lightgreen<ESC>[102m
COLOR bg_yellow<ESC>[103m
COLOR bg_lightblue<ESC>[104m
COLOR bg_lightmagenta<ESC>[105m
COLOR bg_lightcyan<ESC>[106m
COLOR bg_white<ESC>[107m

Both in raw VT100 form and from the COLOR command, multiple colors can be combined. To set yellow text on a blue background in VT100, use "<ESC>[44;33;1m". To set yellow text on a blue background from the COLOR command, use "color bg_blue+yellow".

Using backquotes

Backquotes work by firstly parsing a command line to find matching instances of the "`" character, then executing the command contained between the match and substituting the output of the command into the command line. This is very useful to execute one command to drive the action of another command.

Escaping backquotes

Frequently when creating a command line a choice must be made about when backquote expressions should be executed. By default, these are executed before performing the command line. However, often it is useful to pass the backquote expression to another command, such as for or set. Doing this requires escaping the backquotes, so they are not executed initially but are instead preserved as a string to the next command. As in CMD, Yori uses the "^" character to indicate the following character should be handled literally. For example:

    SET YORIPROMPT=`whoami` $P$$G$
    

Will evaluate "whoami" into a string, then place that string hardcoded into the prompt. However, the prompt can also evaluate the expression every time the prompt is displayed, even if the state changes. Doing this requires escaping to ensure that the SET command receives the backquoted string:

    SET YORIPROMPT=^`whoami^` $P$$G$
    

The same approach applies to environment variables. Consider the following:

    SET YORIPROMPT=%USERNAME% $P$$G$
    SET YORIPROMPT=^%USERNAME^% $P$$G$
    

Modules

Many commands in any shell need to be evaluated within the shell process itself. Some state, such as the current directory or environment, affects the shell process and these commands are only meaningful as in-process commands. Many shells, including CMD.EXE, only support internal commands coded into the shell. Yori allows in-process commands to be added externally and loaded in the form of DLLs with a .COM extension. Any .COM file, when executed, is checked to see if is it a DLL to execute in process, or if it is a regular executable.

Executables in the PATH used in preference to statically provided builtin commands. This is the oppose of CMD's behavior, and is provided so that internal commands can be superseded with enhanced versions in any environment. Every internal command supported by Yori is generated as a DLL as part of the build process, and can be altered or extended as needed.

Modules depend on being able to manipulate process state. In some cases, this can be done by calling the Windows API directly. To interact with functionality provided by the shell, YORI.EXE exports functions which modules can call.

Scripting

Yori does not execute CMD scripts. CMD scripts are executed by CMD. Yori executes its own scripts, which end in a .YS1 extension. When a script is executed, Yori provides extra commands that are not available or meaningful from an interactive command line. These include CALL, GOTO, INCLUDE, RETURN, and SHIFT. Help about these commands is available from "YS /?".

Unlike CMD, a Yori script can invoke another script without terminating execution of the first script (CALL is not required for this.) Environment variables are evaluated when each line is executed (which CMD optionally does via SETLOCAL ENABLEDELAYEDEXPANSION.) Arguments in scripts are referred with training and leading %, as in %1%, whereas CMD uses %1 only. %*% matches all arguments.

Subroutine semantics

Labels within scripts follow the same syntax as CMD, being ":Label" at the beginning of a line. GOTO works similarly to CMD. However, Yori introduces CALL to execute a subroutine, which returns to the call site via the RETURN command. When calling a subroutine, %1% et al refer to arguments passed to the subroutine. Changes within the subroutine are not returned to the calling function by default. The RETURN command specifies which variables should have their changes made to global scope. Consider this simple example:

    CALL subroutine Arg1
    ECHO LOCALVARIABLE=%LOCALVARIABLE% GLOBALVARIABLE=%GLOBALVARIABLE%
    GOTO :eof

    :subroutine
    SET LOCALVARIABLE=%1%
    SET GLOBALVARIABLE=%1%
    RETURN 0 GLOBALVARIABLE
    

In this example, only GLOBALVARIABLE will be set in the main scope, since it was the only value explicitly returned from the subroutine.

CMD script semantics

Yori executes CMD scripts with CMD. However, to provide compatibility with CMD's behavior when executing CMD scripts, it will try to apply changes made by the script to the environment and aliases back to the Yori process. This means that a CMD script containing "SET FOO=BAR" should result in %FOO% being defined in Yori when the script has finished executing.

Updating

Yori and its tools can be updated via the Yori Package Manager, or ypm. In the typical case, running "ypm -u" will update your system to the latest version. Note this requires write access to the Yori binaries, which will be replaced if they are out of date.

Ypm can also be used to install and uninstall packages with a fully qualified file name or URL. It can also search for packages in a list of sources, either local or remote. By default, ypm will use www.malsmith.net as a source. This can be modified by creating a [Sources] section in packages.ini, and adding Source<n>=<URL> lines where n is a monotonically increasing number.

Ypm can also mirror packages, re-routing requests destined to one source to another source. To do this, add a [Mirrors] section in packages.ini with each line containing a value to substitute with another value. The '%' character can be used to indicate an '=' character, which is otherwise inexpressible because it delimits the value to replace with the new value to use. For example, to remap malsmith.net packages to a local location, use:

    [Mirrors]
    http://www.malsmith.net/download/?obj%yori/latest-stable/=c:\local\packages\