Scheduler for VMware
Tanzu 1.3
Scheduler for VMware Tanzu 1.3
Contents
Pivotal Scheduler 5
Overview 5
Product Snapshot 5
Requirements 5
Limitations 5
Scheduler Release Notes 7
Releases 7
v1.3.0 7
Breaking Changes in Scheduler v1.3.0 7
Internal database_source.external.url Property is Secret 7
View Release Notes for Another Version 8
Architecture 9
HA Topology 9
Data Persistence 9
Installing and Configuring Scheduler 10
Prerequisites 10
Download and Install Scheduler 10
Configure Scheduler 10
Configure AZs and Networks 10
Configure Scheduler Options 11
Verify Resource Config 11
Verify Stemcell Version 11
Upgrading to Scheduler v1.3 12
(Optional) Back Up VMware Tanzu SQL with MySQL for VMs Database 12
Upgrade Scheduler to v1.3 with all errands enabled. 12
Using Scheduler 13
Prerequisites 13
Create and Bind a Service Instance Using the cf CLI 13
Confirm Service Availability 13
Scheduler for VMware Tanzu 1.3
VMware, Inc.
3
Create a Service Instance 13
Bind a Service Instance to Your App 14
Manage Jobs and Calls 14
Using Scheduler in Apps Manager 14
Scheduling Jobs 14
Manage Jobs 14
Create a Job 15
Execute a Job 15
Schedule a Job 16
View Jobs 16
View Schedules for Jobs 16
View Job History 16
View Logs 16
Delete a Job 17
Delete a Job Schedule 17
Scheduling Calls 17
Manage Calls 17
Create a Call 18
Execute a Call 18
Schedule a Call 18
View Calls 18
View Schedules for Calls 19
View Call History 19
Delete a Call 19
Delete a Call Schedule 19
Scheduler Cron Syntax 19
Overview 19
Scheduler Cron Syntax 20
Examples of Cron Syntax 20
Quartz Cron Requirements and Limitations 20
Scheduler for VMware Tanzu 1.3
VMware, Inc.
4
Pivotal Scheduler
This documentation describes Pivotal Scheduler.
Overview
Pivotal Scheduler is a service for scheduling the execution of Diego tasks, such as database
migrations, emails, or batch jobs, as well as the execution of outbound HTTP calls.
Pivotal Scheduler enables developers to do the following:
Create, run, and schedule jobs and view job history.
Create, run, and schedule calls and view call history.
You can interact with the service by using the Cloud Foundry Command Line Interface (cf CLI),
Apps Manager, and the Scheduler HTTP API.
Product Snapshot
The following table provides version and version-support information about Pivotal Scheduler.
Element Details
Version v1.2.23
Release date February 21, 2019
Compatible Ops Manager version(s) v2.3.x or later
Compatible Pivotal Application Service version(s)
*
v2.3.x or later
IaaS support AWS, Azure, GCP, OpenStack, and vSphere
Requirements
Pivotal Scheduler requires a MySQL database.
You can use either of the following:
MySQL for Pivotal Platform v2.3 or later. See MySQL for Pivotal Platform.
An external MySQL database
Limitations
If your app uses a buildpack that does not generate a web process type, such as Ruby or
Python, you should do the following:
Scheduler for VMware Tanzu 1.3
VMware, Inc.
5
1. Before pushing your app, create a Procfile in the root directory of the app.
2. Declare a web process type in the file.
If you do not declare this process type, your app will not be accessible through the cf CLI
after you create Scheduler jobs for it.
The maximum number of tasks that you can schedule is determined by the memory and
disk quotas in the Pivotal Scheduler org and space. For more information, see Running
Tasks.
Scheduler for VMware Tanzu 1.3
VMware, Inc.
6
Scheduler Release Notes
This topic contains release notes for Scheduler v1.3.0.
Releases
v1.3.0
Release Date: October 16, 2020
[Security Fix] Updated dependencies mitigate multiple CVEs
[Security Fix] Secrets such as the admin password are scrubbed from the errands
[Feature] CLI now enables specifying a memory and disk limits for jobs so large jobs can
successfully run
[Bug Fix] Increase java scheduler app memory from 1 GB to 2 GB to address problems of
large foundation’s scheduler failing to start
[Bug Fix] Scheduler CLI Plugin now returns data beyond the first page
[Bug Fix] Scheduler no longer incorrectly logs errors caused by jobs that have been
deleted
Breaking Changes in Scheduler v1.3.0
Scheduler v1.3.0 includes the following breaking change:
Internal database_source.external.url Property is Secret
Users of the pivotal-cf/om GitHub repository or Platform Automation will see an error message
similar to the following:
{"errors":{".properties.database_source.external.url":["cannot assign
'mysql://user:password@example.com:3306' type String, expected Object"]}}
To fix the issue, update your product configuration by running:
product-name: p-scheduler
product-properties:
.properties.database_source:
selected_option: external
value: external
.properties.database_source.external.url:
- value: mysql://user:password@example.com:3306
Scheduler for VMware Tanzu 1.3
VMware, Inc.
7
+ value:
+ secret: mysql://user:password@example.com:3306
View Release Notes for Another Version
To view the release notes for another product version, select the version from the dropdown at the
top of this page.
Scheduler for VMware Tanzu 1.3
VMware, Inc.
8
Architecture
This topic describes the architecture of Scheduler.
HA Topology
In a highly available resource configuration, the Scheduler service uses the following:
Three Scheduler instances, each running the Scheduler HTTP API server and a Scheduler
Engine.
Three Scheduler Service Broker instances
You can reduce the number of Scheduler instances to one using the Cloud Foundry Command Line
Interface (cf CLI).
Data Persistence
Scheduler relies on a MySQL datastore to persist data, including jobs, calls, and history. Each call or
schedule can require up to 10 MB of database capacity to store history. A large foundations’s
database disk footprint would be approximately 500 MB.
Scheduler for VMware Tanzu 1.3
VMware, Inc.
9
Installing and Configuring Scheduler
This topic describes how to install and configure Scheduler.
Prerequisites
Before you install Scheduler v1.3, you must have one of the following:
A VMware Tanzu SQL with MySQL for VMs v2.7 or later tile. To download VMware Tanzu
SQL with MySQL for VMs, see VMware Tanzu SQL with MySQL for VMs on VMware Tanzu
Network. For information about installing VMware Tanzu SQL with MySQL for VMs, see
Installing and Configuring VMware Tanzu SQL with MySQL for VMs. For information about
configuring TLS options, see Configure Security.
An external MySQL database with TLS disabled.
Download and Install Scheduler
1. Download the product file from VMware Tanzu Network.
2. Navigate to the Ops Manager Installation Dashboard and click Import a Product to upload
the product file.
3. Under the Import a Product button, click + next to the version number of Scheduler. This
adds the tile to your staging area.
4. Click the newly added Scheduler tile.
Configure Scheduler
Follow the steps below to configure the Scheduler tile.
Configure AZs and Networks
Follow the steps below to choose an Availability Zone (AZ) to run the Scheduler Service Broker and
to select networks.
1. Click Assign AZs and Networks.
2. Configure the fields as follows:
Field Description
Note: Scheduler is compatible with VMware Tanzu SQL with MySQL for VMs.
Compatibility with MySQL alternatives like MariaDB is not tested or guaranteed.
Scheduler for VMware Tanzu 1.3
VMware, Inc.
10
Place
singleton jobs
in
Select the AZ for executing the Scheduler Errands. All Scheduler components execute as
apps and do not require VMs.
Balance other
jobs in
Ignore this field.
Network Select a subnet for the Scheduler Errands. Use the subnet that includes the VMware Tanzu
Application Service for VMs (TAS for VMs) component VMs.
3. Click Save.
Configure Scheduler Options
1. Click Scheduler Configuration.
Enable outbound HTTP calls: This field is enabled by default. Disable this feature to
prevent users from scheduling outbound HTTP requests (calls) from the Scheduler
service. This does not affect outbound connectivity from Diego tasks (jobs).
Database Source > Service Broker:
Name Enter the name of your MySQL service broker. For example, p.mysql.
Plan Name Enter the name of your MySQL service plan, which is used to
provision a database for persistent storage of Scheduler data, including jobs,
calls, and history. For example, db-small.
Database Source > External: Enter a database URL. It must have a scheme, user,
password, host, and port, for example
mysql://user:password@db.example.com:3306. Query parameters are ignored. For
example, with mysql://user:password@db.example.com:3306/?useTLS=false, the
query parameter useTLS is not propagated to Scheduler.
2. Click Save.
Verify Resource Config
1. Click Resource Config.
2. Verify the settings.
3. Click Save.
Verify Stemcell Version
To verify that you have the correct stemcell version for the Scheduler tile, do the following:
1. Click Stemcell.
2. Verify the settings.
3. Download the stemcell from VMware Tanzu Network.
NOTE: The network selected is used only by Errand VMs and does not
apply to the Scheduler runtime.
Scheduler for VMware Tanzu 1.3
VMware, Inc.
11
4. In the Ops Manager, click Stemcell Library.
Stemcell Library
5. Click Import Stemcell, and then select the stemcell you downloaded from VMware Tanzu
Network.
6. Click Save.
7. Return to the Ops Manager Installation Dashboard.
8. Click Review Pending Changes. For more information about this Ops Manager page, see
Reviewing Pending Product Changes.
9. Click Apply Changes.
Upgrading to Scheduler v1.3
This topic explains how to upgrade Scheduler.
(Optional) Back Up VMware Tanzu SQL with MySQL for VMs
Database
VMware recommends backing up your VMware Tanzu SQL with MySQL for VMs database before
installing Scheduler v1.3.
To back up your VMware Tanzu SQL with MySQL for VMs database, see Manual Backup.
Upgrade Scheduler to v1.3 with all errands enabled.
For general information about upgrading a tile, see Upgrading Ops Manager Products.
Note: All Post-Deploy Errands must be enabled to migrate from a MySQL for PCF
v1.x database to VMware Tanzu SQL with MySQL for VMs v2.x database. These
settings are in the Scheduler Errands pane of the Scheduler tile.
Scheduler for VMware Tanzu 1.3
VMware, Inc.
12
Using Scheduler
This topic provides instructions for using Scheduler.
You can interact with the service through the Cloud Foundry Command Line Interface (cf CLI),
Apps Manager, and the Scheduler HTTP API to configure jobs and outbound HTTP calls and to
review history. For general information, see Managing Service Instances with the cf CLI.
Prerequisites
To start using Scheduler, you need the following:
A Tanzu Application Service for VMs (TAS for VMs) deployment with Scheduler installed
and listed in the Marketplace.
A Space Developer account.
(Optional) The cf CLI v6.23.0 or greater and the Scheduler CLI plugin installed on your local
machine. The Scheduler CLI plugin is packaged with the Scheduler tile on VMware Tanzu
Network.
Create and Bind a Service Instance Using the cf CLI
Every app and service in TAS for VMs is scoped to a space. This means that an app can use a
service only if an instance of the service exists in the same space.
The Scheduler service is a singleton service. Only one service instance can be created in a space.
Confirm Service Availability
For apps to use a service, the service must be available in the Marketplace. To confirm the
availability of Scheduler, perform the following steps:
1. Run cf marketplace from the command line.
2. If the output lists scheduler-for-pcf in the service column, Scheduler is available. If the
service is not available, install it. See Installing and Configuring Scheduler for more
information.
Create a Service Instance
$ cf marketplace Getting services from marketplace in org my-org / spa
ce my-space as [email protected]… OK service plans desc
ription […] scheduler-for-pcf standard Scheduler service […]
Scheduler for VMware Tanzu 1.3
VMware, Inc.
13
To create an instance of the Scheduler service, run cf create-service scheduler-for-pcf
standard SERVICE-INSTANCE-NAME, replacing SERVICE-INSTANCE-NAME with a name of your choice.
After you create the service instance, this instance name appears under name in the output of the
cf services command.
See the following example:
You can create only one instance in a space. If you attempt to create more than one instance in a
space, you receive an error response.
Bind a Service Instance to Your App
For an app to use a service, you must bind it to a service instance. Do this after you push or re-
push the app using cf push.
To bind an app to a Scheduler instance, run cf bind-service APP-NAME SERVICE-INSTANCE-NAME,
replacing APP-NAME with the name of the app you want to use the Scheduler service for and
SERVICE-INSTANCE-NAME with the name you provided when you ran cf create-service.
Manage Jobs and Calls
For information about the CLI operations that you can perform to manage jobs and calls in
Scheduler, see Using Jobs and Using Calls.
If you want to manage jobs and calls through the Scheduler HTTP API, see the Scheduler API
Documentation.
Using Scheduler in Apps Manager
For information about binding Scheduler to your app and scheduling tasks through Apps Manager,
see Managing Apps and Service Instances Using Apps Manager.
Scheduling Jobs
This topic provides instructions for managing jobs in Scheduler.
Manage Jobs
$ cf create-service scheduler-for-pcf standard my-instance
Creating service my-instance in org my-org / space my-space as user@example.
com… OK
$ cf services
Getting services in org my-org / space my-space as [email protected]… OK name
service plan bound apps last operation my-instance schedul
er-for-pcf standard create succeeded
$ cf bind-service my-app my-instance
Binding service my-instance to my-app in org my-org / space my-space as user
@example.com… OK TIP: Use ‘cf push’ to ensure your env variable changes take
effect
Scheduler for VMware Tanzu 1.3
VMware, Inc.
14
You can use Scheduler to schedule execution of tasks on Ops Manager runtimes, including
database migrations, emails, and batch jobs. See the following sections to learn more about
creating, running, and scheduling jobs and viewing job history.
To use the Cloud Foundry Command Line Interface (cf CLI) for managing jobs, you must install the
Scheduler CLI plugin on your local machine. This plugin is packaged with the Scheduler tile on
VMware Tanzu Network. For more information, see the Prerequisites section of the Using
Scheduler topic.
Create a Job
To execute a task related to an app, create a job by running the cf create-job APP-NAME JOB-NAME
COMMAND command, where:
APP-NAME is the app you want to execute a task against.
JOB-NAME is the name for your job.
COMMAND is the command you want to execute.
Example:
By default, the job is created with the platform defaults for disk and memory limits. Use the
following command flags to override the default disk and memory limits when you create a job.
--disk LIMIT or -k LIMIT sets the task disk limit.
--memory LIMIT or -m LIMIT sets the task memory limit.
The value of LIMIT must be an integer and be suffixed with either MB for megabytes or GB for
gigabytes. For example, --memory 1024MB sets a memory limit of 1024 megabytes and --disk 5GB
sets a disk limit of 5 gigabytes.
Execute a Job
You can execute a job manually. This is often useful to test the configuration of a job prior to
scheduling it for recurring execution.
Run cf run-job JOB-NAME. See the following example:
Note: Scheduler does not take into account the outcome of previously run jobs. It
assumes that scheduled jobs are idempotent and can be re-run. Jobs that are not
idempotent may cause unintended consequences even if they complete
successfully.
$ cf create-job my-app my-job “pwd”
Creating job my-job for my-app with command pwd in org my-org / space my-spa
ce as [email protected]… Job Name App Name Command my-job my-app
pwd OK
$ cf run-job my-job
Enqueuing job my-job for app my-app in org my-org / space my-space as user@e
xample.com… OK
Scheduler for VMware Tanzu 1.3
VMware, Inc.
15
Schedule a Job
You can schedule a job to execute at any time using a schedule expression. Scheduler requires
Quartz cron expressions in the MIN HOUR DAY-OF-MONTH MONTH DAY-OF-WEEK format. For more
information on how to use this syntax, see Scheduler Cron Syntax.
For example, to execute a job at noon every day, run the following command:
A single job can have multiple schedules. Each schedule has a GUID to distinguish it from similar
schedules.
View Jobs
You can use the cf CLI to list all jobs in a space by running cf jobs. See the following example:
View Schedules for Jobs
You can review schedules for all jobs in a space by running cf job-schedules. See the following
example:
View Job History
You can review job history by running cf job-history JOB-NAME. See the following example:
View Logs
$ cf schedule-job my-job "0 12 ? * * "
Note: Scheduler does not support cron expressions with seconds or years.
Note: Scheduled jobs automatically execute in UTC.
$ cf jobs
Listing jobs for org my-org / space my-space as [email protected]… Job Name
App Name Command my-job my-app pwd OK
$ cf job-schedules
Getting scheduled jobs for org my-org / space my-space as [email protected]…
App Name: my-app my-job pwd 2b69e0c2-9664-46bb-4817-54afcedbb65d
0 12 ? * * OK
$ cf job-history my-job
Getting scheduled job history for my-job in org my-org / space my-space as u
[email protected]… 1 - 1 of 1 Total Results Execution GUID
Execution State Scheduled Time Execution Start Time
Execution End Time Exit Message 8a7e808a5b883a25015b89b4a12c0001
SUCCEEDED Mon, 10 Apr 2017 13:00:00 UTC Mon, 10 Apr 2017 13:00:00 U
TC Mon, 10 Apr 2017 13:00:01 UTC 202 - Cloud Controller Accepted Task
Scheduler for VMware Tanzu 1.3
VMware, Inc.
16
You can view logs for jobs by running cf logs APP-NAME --recent. See the following example:
Delete a Job
You can delete a job by running cf delete-job JOB-NAME. See the following example:
Delete a Job Schedule
You can delete a specific schedule by running cf delete-job-schedule JOB-NAME SCHEDULE-GUID,
where JOB-NAME is the name of the job and SCHEDULE-GUID is the GUID found in the output of the cf
job-schedules command. See the following example:
Scheduling Calls
This topic provides instructions for managing outbound HTTP calls in Scheduler.
Manage Calls
You can use Scheduler to schedule execution of HTTP calls to external HTTP services. See the
following sections to learn more about creating, running, and scheduling calls and viewing call
$ cf logs my-app –recent
Connected, dumping recent logs for app my-app in org my-org / space my-space
as [email protected]… […] 2017-04-19T23:04:13.79-0600 [APP/TASK/cc6fab7f-32a9-
4404-4574-b0c430a96cd9 -|- 0d30f4f0-11a4-4d6a-7e77-5e1cdc1aa5ec/0]OUT Creatin
g container 2017-04-19T23:04:14.01-0600 [APP/TASK/cc6fab7f-32a9-4404-4574-b0c
430a96cd9 -|- 0d30f4f0-11a4-4d6a-7e77-5e1cdc1aa5ec/0]OUT Successfully created
container 2017-04-19T23:04:14.22-0600 [APP/TASK/cc6fab7f-32a9-4404-4574-b0c43
0a96cd9 -|- 0d30f4f0-11a4-4d6a-7e77-5e1cdc1aa5ec/0]OUT bin 2017-04-19T23:04:1
4.22-0600 [APP/TASK/cc6fab7f-32a9-4404-4574-b0c430a96cd9 -|- 0d30f4f0-11a4-4d
6a-7e77-5e1cdc1aa5ec/0]OUT db 2017-04-19T23:04:14.23-0600 [APP/TASK/cc6fab7f-
32a9-4404-4574-b0c430a96cd9 -|- 0d30f4f0-11a4-4d6a-7e77-5e1cdc1aa5ec/0]OUT Ex
it status 0 2017-04-19T23:04:14.24-0600 [APP/TASK/cc6fab7f-32a9-4404-4574-b0c
430a96cd9 -|- 0d30f4f0-11a4-4d6a-7e77-5e1cdc1aa5ec/0]OUT Destroying container
2017-04-19T23:04:14.55-0600 [APP/TASK/cc6fab7f-32a9-4404-4574-b0c430a96cd9 -|
- 0d30f4f0-11a4-4d6a-7e77-5e1cdc1aa5ec/0]OUT Successfully destroyed container
[…]
Note: Scheduler jobs are executed as CF Tasks.
$ cf delete-job my-job
Really delete the job my-job with command pwd and all associated schedules a
nd history?> [yN]:y OK
$ cf delete-job-schedule my-job 2b69e0c2-9664-46bb-4817-54afcedbb65d
Really delete the schedule 2b69e0c2-9664-46bb-4817-54afcedbb65d / 0 12 ? * *
and all associated history?> [yN]: y Deleting schedule 2b69e0c2-9664-46bb-481
7-54afcedbb65d for job my-job in org test / space scheduler as admin OK
Scheduler for VMware Tanzu 1.3
VMware, Inc.
17
history.
To use the Cloud Foundry Command Line Interface (cf CLI) for managing calls, you must install the
Scheduler CLI plugin on your local machine. This plugin is packaged with the Scheduler tile on
VMware Tanzu Network. For more information, see the Prerequisites section of the Using
Scheduler topic.
Create a Call
You can create a call by running the cf create-call APP-NAME CALL-NAME URL command, where:
APP-NAME is the app you want to create a call for.
CALL-NAME is the name for your call.
URL is the URL to execute a HTTP POST call against.
Execute a Call
You can execute a call manually by running the cf run-call CALL-NAME command. This is often
useful to test the configuration of a call prior to scheduling it for recurring execution.
See the following example:
Schedule a Call
You can schedule a call to execute at any time using a schedule expression. Scheduler requires
Quartz cron expressions in the MIN HOUR DAY-OF-MONTH MONTH DAY-OF-WEEK format. For more
information on how to use this syntax, see Scheduler Cron Syntax.
For example, to execute a call at noon every day, run the following command:
A single call can have multiple schedules. Each schedule has a GUID to distinguish it from similar
schedules.
View Calls
You can use the cf CLI to list all calls in a space by running cf calls. See the following example:
$ cf run-call my-call
Enqueuing call my-call for app my-app in org my-org / space my-space as user
@example.com… OK
Note: Call execution must complete within 10 seconds.
$ cf schedule-call my-call "0 12 ? * * "
Note: Scheduled calls automatically execute in UTC.
$ cf calls
Listing calls for org my-org / space my-space as [email protected]… Call Name
App Name URL my-call my-app https://example.com OK
Scheduler for VMware Tanzu 1.3
VMware, Inc.
18
View Schedules for Calls
You can review schedules for all calls in a space by running cf call-schedules. See the following
example:
View Call History
You can review call history by running cf call-history CALL-NAME. See the following example:
Delete a Call
You can delete a call by running cf delete-call CALL-NAME. See the following example:
Delete a Call Schedule
You can delete a specific schedule by running cf delete-call-schedule SCHEDULE-GUID, where
SCHEDULE-GUID is the GUID found in the output of the cf call-schedules command. See the
following example:
Scheduler Cron Syntax
This topic provides instructions for using Quartz cron syntax with Scheduler.
Overview
Scheduler uses Quartz cron, a custom form of cron syntax, for scheduling tasks. This format is
based on traditional cron syntax.
$ cf call-schedules
Getting scheduled calls for org my-org / space my-space as [email protected]…
App Name: my-app my-call 2b69e0c2-9664-46bb-4817-54afcedbb65d 0 12 ? * *
OK
$ cf call-history my-call
Getting scheduled call history for my-call in org my-org / space my-space as
[email protected]… 1 - 1 of 1 Total Results Execution GUID
Execution State Scheduled Time Execution Start Time
Execution End Time Exit Message d288a4ba-e0bc-48c9-969c-6ee79e3
80b20 SUCCEEDED Mon, 16 Oct 2017 12:10:55 UTC Mon, 16 Oct 2017 1
2:10:55 UTC Mon, 16 Oct 2017 12:10:55 UTC 201 - Created
$ cf delete-call my-call
Really delete the call my-call with url https://example.com and all associat
ed schedules and history?> [yN]:y OK
$ cf delete-call-schedule 2b69e0c2-9664-46bb-4817-54afcedbb65d
Really delete the schedule 2b69e0c2-9664-46bb-4817-54afcedbb65d / 0 12 ? * *
and all associated history?> [yN]: y OK
Scheduler for VMware Tanzu 1.3
VMware, Inc.
19
See the sections below for more information about Quartz cron syntax, requirements, and
limitations.
Scheduler Cron Syntax
The following table describes special characters in Quartz cron syntax. You can use these special
characters to create scaling rules with Scheduler.
* Every
? Any
## Value
#-# Range
For more information about special characters, see Cron Trigger Tutorial.
Examples of Cron Syntax
The following are examples of custom scaling rules with Scheduler:
Every Hour:
cf schedule-call my-call "0 * * * ?"
Every day at 8:30am:
cf schedule-call my-call "30 8 ? * *"
Every week at 6pm on Friday:
cf schedule-call my-call "0 18 ? * 5"
For more examples, see Cron Trigger Tutorial.
Quartz Cron Requirements and Limitations
The following are requirements and limitations of Quartz cron:
The day of week cannot conflict with the day of the month in the cron expression. Quartz
cron requires at least one of the “Day” fields to be non-wild. For example, "* * ? * *" and
"* * * * ?" both run the task at the same interval.
Day of week starts at 0 for Sunday.
Scheduler for VMware Tanzu 1.3
VMware, Inc.
20
