<type>documentation</type>
<domain>proj</domain>
<id>clusters</id>
<title>Working with Clusters</title>
<date>$Date: 2010-05-14 11:53:19 -0400 (Fri, 14 May 2010) $</date>
<version>$Revision: 22093 $</version>
<product>pro+</product>

This section describes how to configure Wing to work with a number of containers running
in a cluster managed by a container orchestration system. In this model, a selected set of
containers are debugged, and code may be launched either in context of the running cluster
or in synthesized stand-alone (out-of-cluster) instances of containers.

Configuration
-------------

Projects that use a cluster do so by selecting ``Cluster`` for ``Project Executable`` in
``Project Properties`` and then configuring a cluster.  Cluster configurations may also
be accessed from ``Clusters`` in the ``Project`` menu.

A cluster configuration consists of:

**Identifier** is a unique identifier for the cluster.  This name is used only by
Wing and does not have to match the cluster orchestration system's identifier for
the cluster.

**Type** selects the container orchestration system that manages the cluster. The
available types are supported through the `cluster plugin API <ref:plugins/cluster>`__.

**Configuration** selects the container orchestration system's configuration file
or directory for the selected cluster.

**Main Service** is the main container service in the cluster, to use as the default
location for running processes that are launched out-of-cluster, such as the
``Python Shell`` by default.  This field's menu is not populated until the ``Type`` 
and ``Configuration`` have been set, since those are required to determine which
services exist in the cluster.

**Connect Hostname** is the host name used on containers in the cluster to connect back to
the host where Wing is running. This is used to set up remote inspection of the container,
and to run unit tests, debug processes, and the ``Python Shell``. Docker 18.03 or later
running on Windows and macOS defines ``host.docker.internal`` for this purpose. In other
cases, the IP address for the host system may need to be determined manually. This value
only needs to be valid on the ``Main Service`` container and any containers that will be
debugged.

**Inherit Project Environment** tells Wing to set environment variables defined in
``Project Properties`` into the cluster's container environments. This is off by default
since in most cases containers define their own environment. When any environment
variables are defined in ``Project Properties``, Wing will prompt to explain how these
will be treated in the container. This prompt can be disabled from the dialog or with the
`Project > Containers > Show Environment Warning
<script:show_preferences_gui#prefname="main.show-container-env-warning">`__ preference.

How Debugging Clusters Works
----------------------------

When a project selects a cluster for ``Python Executable``, starting debug from the
``Debug`` menu or toolbar will start the cluster as a whole using an automatically created
derived copy of the ``Configuration`` specified in Wing's cluster configuration. This
derived copy adds some environment variables and file mappings to the services that have
been selected for debug in the `Containers <ref:proj/containers-tool>`__ tool, in order to
cause Python code that is run on those container services to be debugged.

This is accomplished using Python's ``site.py`` capabilities. Wing mounts a Python package
``sitecustomize`` into the user site directory on the container, which is determined by
inspecting Python on the container and obtaining the value of ``site.USER_SITE``.  This
package is automatically loaded by Python at startup and causes Wing's debugger to be
activated by importing ``wingdbstub``.

In order to debug multiple containers at once, the `Debugger > Processes > Enable
Multi-Process Debugging
<script:show_preferences_gui#prefname="debug.multi-process-debug">`__ preference must be
enabled, which it is by default.

In some cases, as determined by the process model used by the code being run on
containers, child process debugging must also be enabled with the `Debug > Processes >
Debug Child Processes
<script:show_preferences_gui#prefname="debug.multi-process-debug-sub-processes">`__
preference or by using the ``Debug Child Processes`` property under the ``Debug/Execute``
tab in ``Project Properties``. For example, when Flask's auto-reload is enabled, child
process debugging must be enabled in Wing, in order to debug the child processes that
Flask spawns to implement reloading.

Container Instance Management
-----------------------------

As noted above, Wing's default debug behavior is to start the whole cluster and debug
selected services. However, Wing can also start debug, unit tests, OS commands, and the
``Python Shell`` in out-of-cluster instances of containers that are defined by the cluster
orchestration system.

Synthesized out-of-cluster instances of the ``Main Service`` selected in Wing's cluster
configuration are used by default for most things, but it is possible to specify not only
whether or not to run processes in-cluster but also which service in the cluster to run
them on. This is done by creating a `launch configuration <ref:proj/launch-configs>`__
from ``Launch Configurations`` in the ``Project`` menu, settings ``Python Executable`` to
``Cluster``, selecting the service to run on, and choosing whether or not to run
in-cluster. The launch configuration can then be used in the following:

**File Properties** accessed by context menu from the editor or ``Project`` tool can
select a particular launch configuration to use for an individual file. This is done by
setting ``Environment`` under the ``Debug/Execute`` tab to ``Use Selected Launch
Configuration`` and selecting the desired launch configuration. The file is then executed
and debugged on the selected cluster service, either in-cluster or out-of-cluster.

**Unit Tests** may be run on a selected service in-cluster or out-of-cluster by setting the
``Environment`` under the ``Testing`` tab in ``Project Properties`` or ``File Properties``
to affect only the environment used when running unit tests.

**OS Commands** may be run on a selected service and in-cluster or out-of-cluster by
setting the ``Execution Context`` under the ``Environment`` tab for ``Command Line`` style
OS commands, by setting the ``File Properties`` on the file used for ``Python File`` style
OS commands, or by selecting a launch configuration for ``Named Entry Point`` style OS
commands.

**Python Shell** processes may be configured run on a particular service, either in-cluster
or out-of-cluster, by selecting a launch configuration under ``Use Environment`` in the
``Python Shell``'s ``Options`` menu.

In contrast to debugging all Python code running on cluster services, these configuration
options make it possible to start the cluster as a whole without debug from the
`Containers tool <ref:proj/containers-tool>`__ and then debug specific files or tests
in-cluster. Which approach to debugging you use depends on the nature of the code running
on your cluster.

Cluster Life Cycle
------------------

When a cluster configuration is edited while the cluster is active, Wing will terminate
any container instances associated with the cluster and restart the ``Python Shell``.
Warning dialogs are displayed before and after a configuration change, unless disabled
from the dialog or with the preferences on the `Project > Containers
<script:show_preferences_gui#prefname="main.show-cluster-config-warning">`__ preferences
page:

**Warn Before Cluster Configuration** controls whether Wing warns before it allows any
changes to a cluster configuration that is in use by running processes.

**Notify Cluster Configuration Change** controls whether Wing notifies that a cluster has
been terminated as a result of a change to the container configuration in Wing Pro.

These same warnings are displayed when starting or stopping a cluster from the ``Containers``
tool.

Note that Wing terminates but does not auto-restart the cluster as a whole. Only
synthesized out-of-cluster service instances are started on demand, for example to run the
``Python Shell``. As a result, for in-cluster execution, you will need to start the
cluster manually from the ``Containers`` tool before starting the in-cluster debugging,
testing, or other processes.

Details and Notes
-----------------

All cluster services started from Wing, even if they are not being debugged, add a mount
of Wing's installation directory into the container at ``/wingpro~~MAJOR_VERSION~~``.  This
is done so that Wing can inspect Python environment and container-only files on
the container, and so that additional in-cluster debug processes may be started
successfully.

To diagnose problems debugging a cluster, set the environment variable WINGDB_LOGFILE for
the services where debug is failing. This can be done in the cluster orchestration
system's configuration or by setting it under ``Environment`` in ``Project Properties``
and enabling ``Inherit Project Environment`` in Wing's cluster configuration. The value
should be ``<stderr>`` for logging into the ``Messages`` tool in Wing or a valid writeable
file path on the container. Contact `support@wingware.com <mailto:support@wingware.com>`__
for help intepreting this output.

