
As compelling as it is, I do not have time to sit and watch long-running data pipelines clock, models train, and python scripts take care of my tasks. Often we want to be sure there wasn’t a failure that requires attention. One solution to this issue is to have an email or text sent for whatever status updates you desire. You can go off and do other tasks without worrying.
There are several good tutorials out there on how to send emails and texts from a python script. Setup will vary depending on if you are working within an organization or on your projects. For this example, I will set up a Twilio phone number and use that account to send myself text updates during a python script run.
After setting up a Twilio account, you access your account id and API token. To send a text through the API, you need to request a Twilio phone number. This entire process takes about 2 minutes. Very fast!
The script below is a typical Machine Learning type of script. In this case, an autoML script runs using the mljar package. I not only want to know when the script has completed running, but I also want to be alerted when there are failures.
# Example of sending text status updates while running an autoML script.
import pandas as pd
# scikit learn utilites
from sklearn.metrics import accuracy_score
from sklearn.model_selection import train_test_split
# mljar-supervised package
from supervised.automl import AutoML
######################################################
# https://www.twilio.com/docs/python/install
from twilio.rest import Client
# Your Account Sid and Auth Token from twilio.com/console
# DANGER! This is insecure. See http://twil.io/secure
account_sid = 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
auth_token = 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
client = Client(account_sid, auth_token)
#######################################################
try:
input_df = pd.read_csv('train.csv')
input_df.fillna(0)
print('Input file columns: ' + str(input_df.columns))
train_data = input_df[['id', 'keyword', 'location', 'text']]
train_label = input_df[['target']]
print('train data : ' + str(train_data.head(1)))
print('train label: ' + str(train_label.head(1)))
X_train, X_test, y_train, y_test = train_test_split(
pd.DataFrame(train_data), train_label, stratify=train_label, test_size=0.25,
random_state=27
)
except:
message = client.messages.create(
body='mljar data load and split failed',
from_='+1xxxxxxxxxx', #twilio #
to='+1xxxxxxxxx'
)
print(message.sid)
try:
automl = AutoML(mode="Explain")
automl.fit(X_train, y_train)
message = client.messages.create(
body='mljar Explain completed',
from_='+xxxxxxxxxx',
to='+1xxxxxxxx'
)
except:
except:
message = client.messages.create(
body='mljar Explain failed',
from_='+xxxxxxxxx',
to='+1xxxxxxxxxx'
Nothing is more disappointing than logging onto your computer after waiting hours for a model to train and finding that it failed hours ago! Prevent this by setting up text or email alerts. It's simple, fast, and is satisfying when you get the 'successful completion' text.
Nothing is more disappointing than logging onto your computer after submitting a script to run and finding that it failed hours ago! Prevent this by setting up text or email alerts. It’s simple, fast, and satisfying when you receive the ‘successful completion’ text.