library(dplyr)
library(ggplot2)
library(prophet)
library(lubridate)

Problema

Exploratorias

El dataset consiste en información real de 8 meses de visitantes de un local de ropa localizado en un shopping de gran tamaño

shopping <- read.csv('local-shopping_prophet.csv') %>% 
  rename(.,visitantes=Total.de.Visitantes) %>%
  mutate(Dia = ymd(Dia))
glimpse(shopping)
Observations: 228
Variables: 3
$ Dia        <date> 2017-09-14, 2017-09-15, 2017-09-16, 2017-09-17, 2017-09-18, 2017-09-19, 2017-09-20, 2017-09-21, 2017-09-22…
$ visitantes <int> 589, 696, 1034, 940, 540, 526, 312, 553, 639, 793, 745, 8, 333, 568, 352, 393, 497, 578, 563, 520, 413, 8, …
$ Clima      <fct> Parcialmente Nublado, Soleado, Soleado, Parcialmente Nublado, Soleado, Soleado, Soleado, Nublado, Soleado, …

Se observa que la serie de tiempo presenta un comportamiento cíclico con algunos eventos atípicos:

  • Aumentos debidos a promociones y Navidad
  • Caídas por cierre del local o caídas del sistema de medición
ggplot(shopping, aes(Dia, visitantes)) + geom_line() + theme_bw() + labs(title='Visitantes por dia')

 ggplot(shopping, aes(Dia, visitantes)) + geom_line() + geom_point(color='forestgreen') + theme_bw() + labs(title='Visitantes por dia')

Técnicas de suavizado

Una primera aproximación puede ser tratar de utilizar técnicas de suavizado.

En este caso estamos utilizando loess como una herramienta gráfica con el comando geom_smooth() de ggplot.

ggplot(shopping, aes(Dia, visitantes)) + geom_point(color='forestgreen') + geom_smooth() + theme_bw() + labs(title='Visitantes por dia: Suavizado')

Si bien está suavizando, el resultado es bastante pobre ya que el modelo no está logrando representar bien la variabilidad de los datos.

Distintas ventanas

Ahora realizamos 4 modelos de LOESS modificando la ventana (span) para cada uno de ellos.

loess=stats::loess(visitantes~as.numeric(Dia), data = shopping, na.action = 'na.exclude', model = T, span=0.01)
shopping['loess']=predict(loess,shopping)
loess_1 = ggplot(shopping, aes(Dia,visitantes)) + geom_point() + geom_line(aes(y=loess), color='firebrick', size=1) + 
  labs(title= "LOESS span:0.01") + theme_bw() 

loess=stats::loess(visitantes~as.numeric(Dia), data = shopping, na.action = 'na.exclude', model = T, span=0.25)
shopping['loess']=predict(loess,shopping)
loess_2 = ggplot(shopping, aes(Dia,visitantes)) + geom_point() + geom_line(aes(y=loess), color='forestgreen', size=1) + 
  labs(title= "LOESS span:0.25") + theme_bw() 

loess=stats::loess(visitantes~as.numeric(Dia), data = shopping, na.action = 'na.exclude', model = T, span=0.5)
shopping['loess']=predict(loess,shopping)
loess_3 = ggplot(shopping, aes(Dia,visitantes)) + geom_point() + geom_line(aes(y=loess), color='steelblue', size=1) + 
  labs(title= "LOESS span:0.50") + theme_bw() 

loess=stats::loess(visitantes~as.numeric(Dia), data = shopping, na.action = 'na.exclude', model = T, span=0.75)
shopping['loess']=predict(loess,shopping)
loess_4 = ggplot(shopping, aes(Dia,visitantes)) + geom_point() + geom_line(aes(y=loess), color='purple', size=1) + 
  labs(title= "LOESS span:0.75") + theme_bw() 

cowplot::plot_grid(loess_1, loess_2, loess_3,loess_4)

Se nota que para un span muy pequeño nuestro modelo va a tener un problema de OVERFITTING: no va a poder generalizar. Mientras que para spans grandes tenemos un problema de UNDERFITTING: nuestro modelo no logra captar bien la variabilidad del fenómeno.

Prophet

Vamos a trabajar sobre la implementación de Prophet en R.

Preparacion del dataset

Prophet requiere que le pasemos el dataset con:

  • ds: la variable temporal

  • y: la variable a predecir

# Eliminamos observaciones con menos de 250 visitantes (cuestion de negocio)
shopping[shopping['visitantes']<250,'visitantes'] = NA
# Creamos el dataset
prophet_df = shopping %>% select(Dia, visitantes) %>% rename(., ds=Dia, y=visitantes)

Modelo

Recordemos que el modelo subyacente es el propuesto por Harvey & Peters (1990):

\(y(t)=g(t)+s(t)+h(t)+\epsilon_t\)

Hay muchos parametros para tener en cuenta. Veremos algunos:

  • df: dataframe

  • growth: tipo de tendencia: lineal o logistica

  • yearly.seasonality: hay estacionalidad anual?

  • yearly.seasonality: hay estacionalidad diaria?

  • holidays: dataframe con fechas de vacaciones/eventos especiales

Modelo básico

La funcion prophet crea el modelo, podemos pasarle o no el dataframe.

La funcion prophet.fit aplica un modelo creado a un dataframe

# Llamamos solo al modelo
prophet_base=prophet()
# Le pasamos el dataset
prophet_base = fit.prophet(m = prophet_base, prophet_df) 
Disabling yearly seasonality. Run prophet with yearly.seasonality=TRUE to override this.
Disabling daily seasonality. Run prophet with daily.seasonality=TRUE to override this.
Initial log joint probability = -4.68957
Optimization terminated normally: 
  Convergence detected: absolute parameter change was below tolerance

Notemos que el modelo automaticamente deshabilita la estacionalidad anual y diaria.

Gráfico del modelo

Llamando a plot obtenemos el valor predicho del modelo y el valor original.

Es dificil de notar pero el modelo realiza predicciones aun para los dias en los cuales no hay datos.

plot(prophet_base,fcst=predict(prophet_base, prophet_df)) +theme_bw()

Componentes del modelo

La funcion prophet_plot_components nos devuelve los efectos de los componentes en nuestra variable a predecir

prophet_plot_components(prophet_base, fcst=predict(prophet_base, prophet_df)) +theme_bw()
NULL

En este caso tenemos la tendencia de los visitantes y la estacionalidad semanal. Esta última parece tener bastante sentido ya que nos muestra que hay mayor cantidad de visitantes durante el fin de semana.

Diagnostico del modelo

La funcion cross_validation permite realizar pronosticos realizando un esquema de cross-validation temporal y, a partir de ellos, obtener ciertas metricas de performance.

  • horizon: horizonte del pronostico. Cuánto tiempo deseo predecir

  • period: periodo entre fechas de analisis. Cuánto voy corriendo mi ventana

  • initial: periodo inicial de entrenamiento

cv_base = cross_validation(prophet_base, initial = 45, period = 7, horizon = 15, units = 'days')
Making 24 forecasts with cutoffs between 2017-11-04 and 2018-04-14
Initial log joint probability = -3.06265
Optimization terminated normally: 
  Convergence detected: absolute parameter change was below tolerance
Initial log joint probability = -2.89179
Optimization terminated normally: 
  Convergence detected: absolute parameter change was below tolerance
Initial log joint probability = -3.42994
Optimization terminated normally: 
  Convergence detected: absolute parameter change was below tolerance
Initial log joint probability = -2.99569
Optimization terminated normally: 
  Convergence detected: absolute parameter change was below tolerance
Initial log joint probability = -3.58187
Optimization terminated normally: 
  Convergence detected: absolute parameter change was below tolerance
Initial log joint probability = -3.40797
Optimization terminated normally: 
  Convergence detected: absolute parameter change was below tolerance
Initial log joint probability = -6.14203
Optimization terminated normally: 
  Convergence detected: absolute parameter change was below tolerance
Initial log joint probability = -6.67306
Optimization terminated normally: 
  Convergence detected: absolute parameter change was below tolerance
Initial log joint probability = -3.38137
Optimization terminated normally: 
  Convergence detected: absolute parameter change was below tolerance
Initial log joint probability = -3.21596
Optimization terminated normally: 
  Convergence detected: absolute parameter change was below tolerance
Initial log joint probability = -3.43862
Optimization terminated normally: 
  Convergence detected: absolute parameter change was below tolerance
Initial log joint probability = -3.36873
Optimization terminated normally: 
  Convergence detected: absolute parameter change was below tolerance
Initial log joint probability = -3.43537
Optimization terminated normally: 
  Convergence detected: absolute parameter change was below tolerance
Initial log joint probability = -3.5117
Optimization terminated normally: 
  Convergence detected: absolute parameter change was below tolerance
Initial log joint probability = -3.60968
Optimization terminated normally: 
  Convergence detected: absolute parameter change was below tolerance
Initial log joint probability = -3.69348
Optimization terminated normally: 
  Convergence detected: absolute parameter change was below tolerance
Initial log joint probability = -3.77909
Optimization terminated normally: 
  Convergence detected: absolute parameter change was below tolerance
Initial log joint probability = -3.8066
Optimization terminated normally: 
  Convergence detected: absolute parameter change was below tolerance
Initial log joint probability = -4.02179
Optimization terminated normally: 
  Convergence detected: absolute parameter change was below tolerance
Initial log joint probability = -4.02246
Optimization terminated normally: 
  Convergence detected: absolute parameter change was below tolerance
Initial log joint probability = -3.99051
Optimization terminated normally: 
  Convergence detected: absolute parameter change was below tolerance
Initial log joint probability = -4.04036
Optimization terminated normally: 
  Convergence detected: relative gradient magnitude is below tolerance
Initial log joint probability = -4.27091
Optimization terminated normally: 
  Convergence detected: absolute parameter change was below tolerance
Initial log joint probability = -4.33551
Optimization terminated normally: 
  Convergence detected: relative gradient magnitude is below tolerance
cv_base

La funcion performance_metrics computa varias metricas de performance a partir de un dataframe de cross validation de prophet

performance_metrics(cv_base, rolling_window = 0.5)

Modelo con estacionalidad mensual

La funcion add_seasonality nos permite agregar nuevas estacionalidades. Las estacionalidades se modelan utilizando series de Fourier que nosotros debemos definir.

Definimos:

  • m: modelo

  • name: nombre de la estacionalidad

  • period: cantidad de dias del periodo

  • fourier.order: orden de la serie de fourier para modelar la estacionalidad

# Llamamos solo al modelo
prophet_mensual=prophet()
# Agregamos la estacionalidad mensual
prophet_mensual=add_seasonality(prophet_mensual, name='monthly', period=365/12, fourier.order = 4)
# Le pasamos el dataset
prophet_mensual = fit.prophet(m = prophet_mensual, prophet_df) 
Disabling yearly seasonality. Run prophet with yearly.seasonality=TRUE to override this.
Disabling daily seasonality. Run prophet with daily.seasonality=TRUE to override this.
Initial log joint probability = -4.68957
Optimization terminated normally: 
  Convergence detected: absolute parameter change was below tolerance

Notemos que sigue deshabilitando la estacionalidad anual y diaria.

Grafico del modelo

plot(prophet_mensual,fcst=predict(prophet_mensual, prophet_df)) +theme_bw()

Componentes del modelo

La funcion prophet_plot_components nos devuelve los efectos de los componentes en nuestra variable a predecir

prophet_plot_components(prophet_mensual, fcst=predict(prophet_mensual, prophet_df)) +theme_bw()
NULL

Notemos que la tendencia y estacionalidad semanal se mantiene y ahora se agrega la tendencia mensual que habíamos incorporado. Nos indica que el ciclo mensual se caracteriza por un crecimiento en la mitad de mes.

Diagnostico del modelo

La funcion performance_metrics computa varias metricas de performance a partir de un dataframe de cross validation de prophet

performance_metrics(cv_mensual, rolling_window = 0.5)

Modelo completo

Como último paso vamos a agregar las ventas de navidad y ciertos dias de promociones como eventos especiales

Dataframe de eventos

Creamos el dataframe de eventos con: nombre del evento, fechas y una “ventana” para definir si el evento se estira a ciertos dias.

# Navidad
christmas = data.frame(holiday= 'christmas',
  ds=ymd(c('2017-12-16','2017-12-17','2017-12-18',
                        '2017-12-19','2017-12-20','2017-12-21',
                        '2017-12-22','2017-12-23')),
  lower_window= 0,
  upper_window= 0)

# Promociones
big_sales = data.frame(
  holiday= 'big_sales',
  ds= ymd(c('2017-09-16','2017-10-08','2017-10-14',
                        '2017-11-20','2017-12-03','2017-12-30')),
  lower_window= 0,
  upper_window= 0)

holidays= bind_rows(christmas, big_sales)
glimpse(holidays)
Observations: 14
Variables: 4
$ holiday      <chr> "christmas", "christmas", "christmas", "christmas", "christmas", "christmas", "christmas", "christmas", "…
$ ds           <date> 2017-12-16, 2017-12-17, 2017-12-18, 2017-12-19, 2017-12-20, 2017-12-21, 2017-12-22, 2017-12-23, 2017-09-…
$ lower_window <dbl> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
$ upper_window <dbl> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
# Llamamos al modelo con el dataset de eventos
prophet_full=prophet(holidays = holidays)
# Agregamos la estacionalidad mensual
prophet_full=add_seasonality(prophet_full, name='monthly', period=30.5, fourier.order = 4)
# Le pasamos el dataset
prophet_full = fit.prophet(m = prophet_full, prophet_df) 
Disabling yearly seasonality. Run prophet with yearly.seasonality=TRUE to override this.
Disabling daily seasonality. Run prophet with daily.seasonality=TRUE to override this.
Initial log joint probability = -4.68957
Optimization terminated normally: 
  Convergence detected: absolute parameter change was below tolerance

Fijense que el modelo automaticamente deshabilita la estacionalidad anual y diaria.

Grafico del modelo

plot(prophet_full,fcst=predict(prophet_full, prophet_df)) +theme_bw()

Componentes del modelo

La funcion prophet_plot_components nos devuelve los efectos de los componentes en nuestra variable a predecir

prophet_plot_components(prophet_full, fcst=predict(prophet_full, prophet_df)) +theme_bw()
NULL

La tendencia y estacionalidad semanal se mantienen aproximadamente igual. La estacionalidad mensual cambia bastante y se observa que existen tres picos en el ciclo mensual. Por su parte, los eventos se modelan como pequeños saltos o picos.

Diagnostico del modelo

performance_metrics(cv_full, rolling_window = 0.5)

Graficos interactivos

Prophet también nos permite realizar gráficos interactivos que suelen ser muy útiles para presentar los resultados.

dyplot.prophet(prophet_full, fcst=predict(prophet_full, prophet_df))
LS0tCnRpdGxlOiAiR0FNIGVuIHNlcmllcyBkZSB0aWVtcG86IFByb3BoZXQiCm91dHB1dDogCiAgaHRtbF9ub3RlYm9vazogCiAgICB0b2M6IHllcwogICAgdG9jX2Zsb2F0OiB5ZXMKYXV0aG9yOiAiSnVhbiBNYW51ZWwgQmFycmlvbGEgeSBEaWVnbyBLb3psb3dza2kiCmRhdGU6IDMwLTExLTIwMTkKLS0tCgo8c3R5bGUgdHlwZT0idGV4dC9jc3MiPgpkaXYubWFpbi1jb250YWluZXIgewogIG1heC13aWR0aDogMTYwMHB4OwogIG1hcmdpbi1sZWZ0OiBhdXRvOwogIG1hcmdpbi1yaWdodDogYXV0bzsKfQo8L3N0eWxlPgoKYGBge3IsIG1lc3NhZ2U9RkFMU0V9CmxpYnJhcnkoZHBseXIpCmxpYnJhcnkoZ2dwbG90MikKbGlicmFyeShwcm9waGV0KQpsaWJyYXJ5KGx1YnJpZGF0ZSkKYGBgCgojIFByb2JsZW1hCgojIyBFeHBsb3JhdG9yaWFzCgpFbCBkYXRhc2V0IGNvbnNpc3RlIGVuIGluZm9ybWFjacOzbiByZWFsIGRlIDggbWVzZXMgZGUgdmlzaXRhbnRlcyBkZSB1biBsb2NhbCBkZSByb3BhIGxvY2FsaXphZG8gZW4gdW4gc2hvcHBpbmcgZGUgZ3JhbiB0YW1hw7FvCgpgYGB7cn0Kc2hvcHBpbmcgPC0gcmVhZC5jc3YoJ2xvY2FsLXNob3BwaW5nX3Byb3BoZXQuY3N2JykgJT4lIAogIHJlbmFtZSguLHZpc2l0YW50ZXM9VG90YWwuZGUuVmlzaXRhbnRlcykgJT4lCiAgbXV0YXRlKERpYSA9IHltZChEaWEpKQpnbGltcHNlKHNob3BwaW5nKQpgYGAKClNlIG9ic2VydmEgcXVlIGxhIHNlcmllIGRlIHRpZW1wbyBwcmVzZW50YSB1biBjb21wb3J0YW1pZW50byBjw61jbGljbyBjb24gYWxndW5vcyBldmVudG9zIGF0w61waWNvczoKCiogQXVtZW50b3MgZGViaWRvcyBhIHByb21vY2lvbmVzIHkgTmF2aWRhZAoqIENhw61kYXMgcG9yIGNpZXJyZSBkZWwgbG9jYWwgbyBjYcOtZGFzIGRlbCBzaXN0ZW1hIGRlIG1lZGljacOzbgoKYGBge3J9CmdncGxvdChzaG9wcGluZywgYWVzKERpYSwgdmlzaXRhbnRlcykpICsgZ2VvbV9saW5lKCkgKyB0aGVtZV9idygpICsgbGFicyh0aXRsZT0nVmlzaXRhbnRlcyBwb3IgZGlhJykKYGBgCgpgYGB7cn0KIGdncGxvdChzaG9wcGluZywgYWVzKERpYSwgdmlzaXRhbnRlcykpICsgZ2VvbV9saW5lKCkgKyBnZW9tX3BvaW50KGNvbG9yPSdmb3Jlc3RncmVlbicpICsgdGhlbWVfYncoKSArIGxhYnModGl0bGU9J1Zpc2l0YW50ZXMgcG9yIGRpYScpCmBgYAoKIyBUw6ljbmljYXMgZGUgc3Vhdml6YWRvCgpVbmEgcHJpbWVyYSBhcHJveGltYWNpw7NuIHB1ZWRlIHNlciB0cmF0YXIgZGUgdXRpbGl6YXIgdMOpY25pY2FzIGRlIHN1YXZpemFkby4KCkVuIGVzdGUgY2FzbyBlc3RhbW9zIHV0aWxpemFuZG8gKipsb2VzcyoqIGNvbW8gdW5hIGhlcnJhbWllbnRhIGdyw6FmaWNhIGNvbiBlbCBjb21hbmRvIGBnZW9tX3Ntb290aCgpYCBkZSBnZ3Bsb3QuCgpgYGB7cn0KZ2dwbG90KHNob3BwaW5nLCBhZXMoRGlhLCB2aXNpdGFudGVzKSkgKyBnZW9tX3BvaW50KGNvbG9yPSdmb3Jlc3RncmVlbicpICsgZ2VvbV9zbW9vdGgoKSArIHRoZW1lX2J3KCkgKyBsYWJzKHRpdGxlPSdWaXNpdGFudGVzIHBvciBkaWE6IFN1YXZpemFkbycpCmBgYAoKU2kgYmllbiBlc3TDoSBzdWF2aXphbmRvLCBlbCByZXN1bHRhZG8gZXMgYmFzdGFudGUgcG9icmUgeWEgcXVlIGVsIG1vZGVsbyBubyBlc3TDoSBsb2dyYW5kbyByZXByZXNlbnRhciBiaWVuIGxhIHZhcmlhYmlsaWRhZCBkZSBsb3MgZGF0b3MuCgojIyBEaXN0aW50YXMgdmVudGFuYXMKCkFob3JhIHJlYWxpemFtb3MgNCBtb2RlbG9zIGRlIExPRVNTIG1vZGlmaWNhbmRvIGxhIHZlbnRhbmEgKCoqc3BhbioqKSBwYXJhIGNhZGEgdW5vIGRlIGVsbG9zLgoKYGBge3IsIHdhcm5pbmc9RkFMU0V9CmxvZXNzPXN0YXRzOjpsb2Vzcyh2aXNpdGFudGVzfmFzLm51bWVyaWMoRGlhKSwgZGF0YSA9IHNob3BwaW5nLCBuYS5hY3Rpb24gPSAnbmEuZXhjbHVkZScsIG1vZGVsID0gVCwgc3Bhbj0wLjAxKQpzaG9wcGluZ1snbG9lc3MnXT1wcmVkaWN0KGxvZXNzLHNob3BwaW5nKQpsb2Vzc18xID0gZ2dwbG90KHNob3BwaW5nLCBhZXMoRGlhLHZpc2l0YW50ZXMpKSArIGdlb21fcG9pbnQoKSArIGdlb21fbGluZShhZXMoeT1sb2VzcyksIGNvbG9yPSdmaXJlYnJpY2snLCBzaXplPTEpICsgCiAgbGFicyh0aXRsZT0gIkxPRVNTIHNwYW46MC4wMSIpICsgdGhlbWVfYncoKSAKCmxvZXNzPXN0YXRzOjpsb2Vzcyh2aXNpdGFudGVzfmFzLm51bWVyaWMoRGlhKSwgZGF0YSA9IHNob3BwaW5nLCBuYS5hY3Rpb24gPSAnbmEuZXhjbHVkZScsIG1vZGVsID0gVCwgc3Bhbj0wLjI1KQpzaG9wcGluZ1snbG9lc3MnXT1wcmVkaWN0KGxvZXNzLHNob3BwaW5nKQpsb2Vzc18yID0gZ2dwbG90KHNob3BwaW5nLCBhZXMoRGlhLHZpc2l0YW50ZXMpKSArIGdlb21fcG9pbnQoKSArIGdlb21fbGluZShhZXMoeT1sb2VzcyksIGNvbG9yPSdmb3Jlc3RncmVlbicsIHNpemU9MSkgKyAKICBsYWJzKHRpdGxlPSAiTE9FU1Mgc3BhbjowLjI1IikgKyB0aGVtZV9idygpIAoKbG9lc3M9c3RhdHM6OmxvZXNzKHZpc2l0YW50ZXN+YXMubnVtZXJpYyhEaWEpLCBkYXRhID0gc2hvcHBpbmcsIG5hLmFjdGlvbiA9ICduYS5leGNsdWRlJywgbW9kZWwgPSBULCBzcGFuPTAuNSkKc2hvcHBpbmdbJ2xvZXNzJ109cHJlZGljdChsb2VzcyxzaG9wcGluZykKbG9lc3NfMyA9IGdncGxvdChzaG9wcGluZywgYWVzKERpYSx2aXNpdGFudGVzKSkgKyBnZW9tX3BvaW50KCkgKyBnZW9tX2xpbmUoYWVzKHk9bG9lc3MpLCBjb2xvcj0nc3RlZWxibHVlJywgc2l6ZT0xKSArIAogIGxhYnModGl0bGU9ICJMT0VTUyBzcGFuOjAuNTAiKSArIHRoZW1lX2J3KCkgCgpsb2Vzcz1zdGF0czo6bG9lc3ModmlzaXRhbnRlc35hcy5udW1lcmljKERpYSksIGRhdGEgPSBzaG9wcGluZywgbmEuYWN0aW9uID0gJ25hLmV4Y2x1ZGUnLCBtb2RlbCA9IFQsIHNwYW49MC43NSkKc2hvcHBpbmdbJ2xvZXNzJ109cHJlZGljdChsb2VzcyxzaG9wcGluZykKbG9lc3NfNCA9IGdncGxvdChzaG9wcGluZywgYWVzKERpYSx2aXNpdGFudGVzKSkgKyBnZW9tX3BvaW50KCkgKyBnZW9tX2xpbmUoYWVzKHk9bG9lc3MpLCBjb2xvcj0ncHVycGxlJywgc2l6ZT0xKSArIAogIGxhYnModGl0bGU9ICJMT0VTUyBzcGFuOjAuNzUiKSArIHRoZW1lX2J3KCkgCgpjb3dwbG90OjpwbG90X2dyaWQobG9lc3NfMSwgbG9lc3NfMiwgbG9lc3NfMyxsb2Vzc180KQpgYGAKClNlIG5vdGEgcXVlIHBhcmEgdW4gKipzcGFuIG11eSBwZXF1ZcOxbyoqIG51ZXN0cm8gbW9kZWxvIHZhIGEgdGVuZXIgdW4gcHJvYmxlbWEgZGUgKipPVkVSRklUVElORyoqOiBubyB2YSBhIHBvZGVyIGdlbmVyYWxpemFyLiBNaWVudHJhcyBxdWUgcGFyYSAqKnNwYW5zIGdyYW5kZXMqKiB0ZW5lbW9zIHVuIHByb2JsZW1hIGRlICoqVU5ERVJGSVRUSU5HKio6IG51ZXN0cm8gbW9kZWxvIG5vIGxvZ3JhIGNhcHRhciBiaWVuIGxhIHZhcmlhYmlsaWRhZCBkZWwgZmVuw7NtZW5vLgoKIyBQcm9waGV0CgpWYW1vcyBhIHRyYWJhamFyIHNvYnJlIGxhIGltcGxlbWVudGFjacOzbiBkZSBQcm9waGV0IGVuIFIuCgojIyBQcmVwYXJhY2lvbiBkZWwgZGF0YXNldAoKUHJvcGhldCByZXF1aWVyZSBxdWUgbGUgcGFzZW1vcyBlbCBkYXRhc2V0IGNvbjoKCiAgKiAqKmRzKio6IGxhIHZhcmlhYmxlIHRlbXBvcmFsIAoKICAqICoqeSoqOiBsYSB2YXJpYWJsZSBhIHByZWRlY2lyCgpgYGB7cn0KIyBFbGltaW5hbW9zIG9ic2VydmFjaW9uZXMgY29uIG1lbm9zIGRlIDI1MCB2aXNpdGFudGVzIChjdWVzdGlvbiBkZSBuZWdvY2lvKQpzaG9wcGluZ1tzaG9wcGluZ1sndmlzaXRhbnRlcyddPDI1MCwndmlzaXRhbnRlcyddID0gTkEKIyBDcmVhbW9zIGVsIGRhdGFzZXQKcHJvcGhldF9kZiA9IHNob3BwaW5nICU+JSBzZWxlY3QoRGlhLCB2aXNpdGFudGVzKSAlPiUgcmVuYW1lKC4sIGRzPURpYSwgeT12aXNpdGFudGVzKQpgYGAKCiMjIE1vZGVsbwoKUmVjb3JkZW1vcyBxdWUgZWwgbW9kZWxvIHN1YnlhY2VudGUgZXMgZWwgcHJvcHVlc3RvIHBvciBIYXJ2ZXkgJiBQZXRlcnMgKDE5OTApOiAKCiR5KHQpPWcodCkrcyh0KStoKHQpK1xlcHNpbG9uX3QkCgpIYXkgbXVjaG9zIHBhcmFtZXRyb3MgcGFyYSB0ZW5lciBlbiBjdWVudGEuIFZlcmVtb3MgYWxndW5vczoKCiAgKiAqKmRmKio6IGRhdGFmcmFtZQogIAogICogKipncm93dGgqKjogdGlwbyBkZSB0ZW5kZW5jaWE6IGxpbmVhbCBvIGxvZ2lzdGljYQoKICAqICoqeWVhcmx5LnNlYXNvbmFsaXR5Kio6IGhheSBlc3RhY2lvbmFsaWRhZCBhbnVhbD8KICAKICAqICoqeWVhcmx5LnNlYXNvbmFsaXR5Kio6IGhheSBlc3RhY2lvbmFsaWRhZCBkaWFyaWE/CiAgCiAgKiAgKipob2xpZGF5cyoqOiBkYXRhZnJhbWUgY29uIGZlY2hhcyBkZSB2YWNhY2lvbmVzL2V2ZW50b3MgZXNwZWNpYWxlcwoKIyMjIE1vZGVsbyBiw6FzaWNvCgpMYSBmdW5jaW9uIGBwcm9waGV0YCBjcmVhIGVsIG1vZGVsbywgcG9kZW1vcyBwYXNhcmxlIG8gbm8gZWwgZGF0YWZyYW1lLgoKTGEgZnVuY2lvbiBgcHJvcGhldC5maXRgIGFwbGljYSB1biBtb2RlbG8gY3JlYWRvIGEgdW4gZGF0YWZyYW1lCgpgYGB7cn0KIyBMbGFtYW1vcyBzb2xvIGFsIG1vZGVsbwpwcm9waGV0X2Jhc2U9cHJvcGhldCgpCiMgTGUgcGFzYW1vcyBlbCBkYXRhc2V0CnByb3BoZXRfYmFzZSA9IGZpdC5wcm9waGV0KG0gPSBwcm9waGV0X2Jhc2UsIHByb3BoZXRfZGYpIApgYGAKCk5vdGVtb3MgcXVlIGVsIG1vZGVsbyBhdXRvbWF0aWNhbWVudGUgZGVzaGFiaWxpdGEgbGEgZXN0YWNpb25hbGlkYWQgYW51YWwgeSBkaWFyaWEuCgojIyMjIEdyw6FmaWNvIGRlbCBtb2RlbG8KCkxsYW1hbmRvIGEgYHBsb3RgIG9idGVuZW1vcyBlbCB2YWxvciBwcmVkaWNobyBkZWwgbW9kZWxvIHkgZWwgdmFsb3Igb3JpZ2luYWwuCgpFcyBkaWZpY2lsIGRlIG5vdGFyIHBlcm8gZWwgbW9kZWxvIHJlYWxpemEgcHJlZGljY2lvbmVzIGF1biBwYXJhIGxvcyBkaWFzIGVuIGxvcyBjdWFsZXMgbm8gaGF5IGRhdG9zLgoKYGBge3J9CnBsb3QocHJvcGhldF9iYXNlLGZjc3Q9cHJlZGljdChwcm9waGV0X2Jhc2UsIHByb3BoZXRfZGYpKSArdGhlbWVfYncoKQpgYGAKCiMjIyMgQ29tcG9uZW50ZXMgZGVsIG1vZGVsbwoKTGEgZnVuY2lvbiBgcHJvcGhldF9wbG90X2NvbXBvbmVudHNgIG5vcyBkZXZ1ZWx2ZSBsb3MgZWZlY3RvcyBkZSBsb3MgY29tcG9uZW50ZXMgZW4gbnVlc3RyYSB2YXJpYWJsZSBhIHByZWRlY2lyCgpgYGB7cn0KcHJvcGhldF9wbG90X2NvbXBvbmVudHMocHJvcGhldF9iYXNlLCBmY3N0PXByZWRpY3QocHJvcGhldF9iYXNlLCBwcm9waGV0X2RmKSkgK3RoZW1lX2J3KCkKYGBgCgpFbiBlc3RlIGNhc28gdGVuZW1vcyBsYSB0ZW5kZW5jaWEgZGUgbG9zIHZpc2l0YW50ZXMgeSBsYSBlc3RhY2lvbmFsaWRhZCBzZW1hbmFsLiBFc3RhIMO6bHRpbWEgcGFyZWNlIHRlbmVyIGJhc3RhbnRlIHNlbnRpZG8geWEgcXVlIG5vcyBtdWVzdHJhIHF1ZSBoYXkgbWF5b3IgY2FudGlkYWQgZGUgdmlzaXRhbnRlcyBkdXJhbnRlIGVsIGZpbiBkZSBzZW1hbmEuCgojIyMjIERpYWdub3N0aWNvIGRlbCBtb2RlbG8KCkxhIGZ1bmNpb24gYGNyb3NzX3ZhbGlkYXRpb25gIHBlcm1pdGUgcmVhbGl6YXIgcHJvbm9zdGljb3MgcmVhbGl6YW5kbyB1biBlc3F1ZW1hIGRlIGNyb3NzLXZhbGlkYXRpb24gdGVtcG9yYWwgeSwgYSBwYXJ0aXIgZGUgZWxsb3MsIG9idGVuZXIgY2llcnRhcyBtZXRyaWNhcyBkZSBwZXJmb3JtYW5jZS4KCiAgKiAqKmhvcml6b24qKjogaG9yaXpvbnRlIGRlbCBwcm9ub3N0aWNvLiBDdcOhbnRvIHRpZW1wbyBkZXNlbyBwcmVkZWNpcgogIAogICogKipwZXJpb2QqKjogcGVyaW9kbyBlbnRyZSBmZWNoYXMgZGUgYW5hbGlzaXMuIEN1w6FudG8gdm95IGNvcnJpZW5kbyBtaSB2ZW50YW5hCiAgCiAgICogKippbml0aWFsKio6IHBlcmlvZG8gaW5pY2lhbCBkZSBlbnRyZW5hbWllbnRvCgpgYGB7cn0KY3ZfYmFzZSA9IGNyb3NzX3ZhbGlkYXRpb24ocHJvcGhldF9iYXNlLCBpbml0aWFsID0gNDUsIHBlcmlvZCA9IDcsIGhvcml6b24gPSAxNSwgdW5pdHMgPSAnZGF5cycpCmN2X2Jhc2UKYGBgCgpMYSBmdW5jaW9uIGBwZXJmb3JtYW5jZV9tZXRyaWNzYCBjb21wdXRhIHZhcmlhcyBtZXRyaWNhcyBkZSBwZXJmb3JtYW5jZSBhIHBhcnRpciBkZSB1biBkYXRhZnJhbWUgZGUgY3Jvc3MgdmFsaWRhdGlvbiBkZSBwcm9waGV0CgpgYGB7cn0KcGVyZm9ybWFuY2VfbWV0cmljcyhjdl9iYXNlLCByb2xsaW5nX3dpbmRvdyA9IDAuNSkKYGBgCgojIyMgTW9kZWxvIGNvbiBlc3RhY2lvbmFsaWRhZCBtZW5zdWFsCgpMYSBmdW5jaW9uIGBhZGRfc2Vhc29uYWxpdHlgIG5vcyBwZXJtaXRlIGFncmVnYXIgbnVldmFzIGVzdGFjaW9uYWxpZGFkZXMuIExhcyBlc3RhY2lvbmFsaWRhZGVzIHNlIG1vZGVsYW4gdXRpbGl6YW5kbyBzZXJpZXMgZGUgRm91cmllciBxdWUgbm9zb3Ryb3MgZGViZW1vcyBkZWZpbmlyLgoKRGVmaW5pbW9zOgoKICAqICoqbSoqOiBtb2RlbG8KICAKICAqICoqbmFtZSoqOiBub21icmUgZGUgbGEgZXN0YWNpb25hbGlkYWQKICAKICAqICoqcGVyaW9kKio6IGNhbnRpZGFkIGRlIGRpYXMgZGVsIHBlcmlvZG8KICAKICAqICoqZm91cmllci5vcmRlcioqOiBvcmRlbiBkZSBsYSBzZXJpZSBkZSBmb3VyaWVyIHBhcmEgbW9kZWxhciBsYSBlc3RhY2lvbmFsaWRhZAoKYGBge3J9CiMgTGxhbWFtb3Mgc29sbyBhbCBtb2RlbG8KcHJvcGhldF9tZW5zdWFsPXByb3BoZXQoKQojIEFncmVnYW1vcyBsYSBlc3RhY2lvbmFsaWRhZCBtZW5zdWFsCnByb3BoZXRfbWVuc3VhbD1hZGRfc2Vhc29uYWxpdHkocHJvcGhldF9tZW5zdWFsLCBuYW1lPSdtb250aGx5JywgcGVyaW9kPTM2NS8xMiwgZm91cmllci5vcmRlciA9IDQpCiMgTGUgcGFzYW1vcyBlbCBkYXRhc2V0CnByb3BoZXRfbWVuc3VhbCA9IGZpdC5wcm9waGV0KG0gPSBwcm9waGV0X21lbnN1YWwsIHByb3BoZXRfZGYpIApgYGAKTm90ZW1vcyBxdWUgc2lndWUgZGVzaGFiaWxpdGFuZG8gbGEgZXN0YWNpb25hbGlkYWQgYW51YWwgeSBkaWFyaWEuCgojIyMjIEdyYWZpY28gZGVsIG1vZGVsbwoKYGBge3J9CnBsb3QocHJvcGhldF9tZW5zdWFsLGZjc3Q9cHJlZGljdChwcm9waGV0X21lbnN1YWwsIHByb3BoZXRfZGYpKSArdGhlbWVfYncoKQpgYGAKCiMjIyMgQ29tcG9uZW50ZXMgZGVsIG1vZGVsbwoKTGEgZnVuY2lvbiBgcHJvcGhldF9wbG90X2NvbXBvbmVudHNgIG5vcyBkZXZ1ZWx2ZSBsb3MgZWZlY3RvcyBkZSBsb3MgY29tcG9uZW50ZXMgZW4gbnVlc3RyYSB2YXJpYWJsZSBhIHByZWRlY2lyCgpgYGB7cn0KcHJvcGhldF9wbG90X2NvbXBvbmVudHMocHJvcGhldF9tZW5zdWFsLCBmY3N0PXByZWRpY3QocHJvcGhldF9tZW5zdWFsLCBwcm9waGV0X2RmKSkgK3RoZW1lX2J3KCkKYGBgCgpOb3RlbW9zIHF1ZSBsYSB0ZW5kZW5jaWEgeSBlc3RhY2lvbmFsaWRhZCBzZW1hbmFsIHNlIG1hbnRpZW5lIHkgYWhvcmEgc2UgYWdyZWdhIGxhIHRlbmRlbmNpYSBtZW5zdWFsIHF1ZSBoYWLDrWFtb3MgaW5jb3Jwb3JhZG8uIE5vcyBpbmRpY2EgcXVlIGVsIGNpY2xvIG1lbnN1YWwgc2UgY2FyYWN0ZXJpemEgcG9yIHVuIGNyZWNpbWllbnRvIGVuIGxhIG1pdGFkIGRlIG1lcy4KCiMjIyMgRGlhZ25vc3RpY28gZGVsIG1vZGVsbwoKYGBge3IsIG1lc3NhZ2U9RkFMU0UsIGVjaG89RkFMU0UsIHJlc3VsdHM9J2hpZGUnfQpjdl9tZW5zdWFsID0gY3Jvc3NfdmFsaWRhdGlvbihwcm9waGV0X21lbnN1YWwsIGluaXRpYWwgPSA0NSwgcGVyaW9kID0gNywgaG9yaXpvbiA9IDE1LCB1bml0cyA9ICdkYXlzJykKY3ZfbWVuc3VhbApgYGAKCkxhIGZ1bmNpb24gYHBlcmZvcm1hbmNlX21ldHJpY3NgIGNvbXB1dGEgdmFyaWFzIG1ldHJpY2FzIGRlIHBlcmZvcm1hbmNlIGEgcGFydGlyIGRlIHVuIGRhdGFmcmFtZSBkZSBjcm9zcyB2YWxpZGF0aW9uIGRlIHByb3BoZXQKCmBgYHtyfQpwZXJmb3JtYW5jZV9tZXRyaWNzKGN2X21lbnN1YWwsIHJvbGxpbmdfd2luZG93ID0gMC41KQpgYGAKCiMjIyBNb2RlbG8gY29tcGxldG8gCgpDb21vIMO6bHRpbW8gcGFzbyB2YW1vcyBhIGFncmVnYXIgbGFzIHZlbnRhcyBkZSBuYXZpZGFkIHkgY2llcnRvcyBkaWFzIGRlIHByb21vY2lvbmVzIGNvbW8gZXZlbnRvcyBlc3BlY2lhbGVzCgojIyMjIERhdGFmcmFtZSBkZSBldmVudG9zCgpDcmVhbW9zIGVsIGRhdGFmcmFtZSBkZSBldmVudG9zIGNvbjogbm9tYnJlIGRlbCBldmVudG8sIGZlY2hhcyB5IHVuYSAidmVudGFuYSIgcGFyYSBkZWZpbmlyIHNpIGVsIGV2ZW50byBzZSBlc3RpcmEgYSBjaWVydG9zIGRpYXMuCgpgYGB7ciwgd2FybmluZz1GQUxTRX0KIyBOYXZpZGFkCmNocmlzdG1hcyA9IGRhdGEuZnJhbWUoaG9saWRheT0gJ2NocmlzdG1hcycsCiAgZHM9eW1kKGMoJzIwMTctMTItMTYnLCcyMDE3LTEyLTE3JywnMjAxNy0xMi0xOCcsCiAgICAgICAgICAgICAgICAgICAgICAgICcyMDE3LTEyLTE5JywnMjAxNy0xMi0yMCcsJzIwMTctMTItMjEnLAogICAgICAgICAgICAgICAgICAgICAgICAnMjAxNy0xMi0yMicsJzIwMTctMTItMjMnKSksCiAgbG93ZXJfd2luZG93PSAwLAogIHVwcGVyX3dpbmRvdz0gMCkKCiMgUHJvbW9jaW9uZXMKYmlnX3NhbGVzID0gZGF0YS5mcmFtZSgKICBob2xpZGF5PSAnYmlnX3NhbGVzJywKICBkcz0geW1kKGMoJzIwMTctMDktMTYnLCcyMDE3LTEwLTA4JywnMjAxNy0xMC0xNCcsCiAgICAgICAgICAgICAgICAgICAgICAgICcyMDE3LTExLTIwJywnMjAxNy0xMi0wMycsJzIwMTctMTItMzAnKSksCiAgbG93ZXJfd2luZG93PSAwLAogIHVwcGVyX3dpbmRvdz0gMCkKCmhvbGlkYXlzPSBiaW5kX3Jvd3MoY2hyaXN0bWFzLCBiaWdfc2FsZXMpCmdsaW1wc2UoaG9saWRheXMpCmBgYAoKYGBge3J9CiMgTGxhbWFtb3MgYWwgbW9kZWxvIGNvbiBlbCBkYXRhc2V0IGRlIGV2ZW50b3MKcHJvcGhldF9mdWxsPXByb3BoZXQoaG9saWRheXMgPSBob2xpZGF5cykKIyBBZ3JlZ2Ftb3MgbGEgZXN0YWNpb25hbGlkYWQgbWVuc3VhbApwcm9waGV0X2Z1bGw9YWRkX3NlYXNvbmFsaXR5KHByb3BoZXRfZnVsbCwgbmFtZT0nbW9udGhseScsIHBlcmlvZD0zMC41LCBmb3VyaWVyLm9yZGVyID0gNCkKIyBMZSBwYXNhbW9zIGVsIGRhdGFzZXQKcHJvcGhldF9mdWxsID0gZml0LnByb3BoZXQobSA9IHByb3BoZXRfZnVsbCwgcHJvcGhldF9kZikgCmBgYApGaWplbnNlIHF1ZSBlbCBtb2RlbG8gYXV0b21hdGljYW1lbnRlIGRlc2hhYmlsaXRhIGxhIGVzdGFjaW9uYWxpZGFkIGFudWFsIHkgZGlhcmlhLgoKIyMjIyBHcmFmaWNvIGRlbCBtb2RlbG8KCmBgYHtyfQpwbG90KHByb3BoZXRfZnVsbCxmY3N0PXByZWRpY3QocHJvcGhldF9mdWxsLCBwcm9waGV0X2RmKSkgK3RoZW1lX2J3KCkKYGBgCgojIyMjIENvbXBvbmVudGVzIGRlbCBtb2RlbG8KCkxhIGZ1bmNpb24gYHByb3BoZXRfcGxvdF9jb21wb25lbnRzYCBub3MgZGV2dWVsdmUgbG9zIGVmZWN0b3MgZGUgbG9zIGNvbXBvbmVudGVzIGVuIG51ZXN0cmEgdmFyaWFibGUgYSBwcmVkZWNpcgoKYGBge3J9CnByb3BoZXRfcGxvdF9jb21wb25lbnRzKHByb3BoZXRfZnVsbCwgZmNzdD1wcmVkaWN0KHByb3BoZXRfZnVsbCwgcHJvcGhldF9kZikpICt0aGVtZV9idygpCmBgYAoKTGEgdGVuZGVuY2lhIHkgZXN0YWNpb25hbGlkYWQgc2VtYW5hbCBzZSBtYW50aWVuZW4gYXByb3hpbWFkYW1lbnRlIGlndWFsLiBMYSBlc3RhY2lvbmFsaWRhZCBtZW5zdWFsIGNhbWJpYSBiYXN0YW50ZSB5IHNlIG9ic2VydmEgcXVlIGV4aXN0ZW4gdHJlcyBwaWNvcyBlbiBlbCBjaWNsbyBtZW5zdWFsLiBQb3Igc3UgcGFydGUsIGxvcyBldmVudG9zIHNlIG1vZGVsYW4gY29tbyBwZXF1ZcOxb3Mgc2FsdG9zIG8gcGljb3MuCgojIyMjIERpYWdub3N0aWNvIGRlbCBtb2RlbG8KCmBgYHtyLCBtZXNzYWdlPUZBTFNFLCBlY2hvPUZBTFNFLCByZXN1bHRzPSdoaWRlJ30KY3ZfZnVsbCA9IGNyb3NzX3ZhbGlkYXRpb24ocHJvcGhldF9mdWxsLCBpbml0aWFsID0gNDUsIHBlcmlvZCA9IDcsIGhvcml6b24gPSAxNSwgdW5pdHMgPSAnZGF5cycpCmN2X2Z1bGwKYGBgCgoKYGBge3J9CnBlcmZvcm1hbmNlX21ldHJpY3MoY3ZfZnVsbCwgcm9sbGluZ193aW5kb3cgPSAwLjUpCmBgYAoKIyMgR3JhZmljb3MgaW50ZXJhY3Rpdm9zCgpQcm9waGV0IHRhbWJpw6luIG5vcyBwZXJtaXRlIHJlYWxpemFyIGdyw6FmaWNvcyBpbnRlcmFjdGl2b3MgcXVlIHN1ZWxlbiBzZXIgbXV5IMO6dGlsZXMgcGFyYSBwcmVzZW50YXIgbG9zIHJlc3VsdGFkb3MuCgpgYGB7ciwgZmlnLndpZHRoPTgsZmlnLmhlaWdodD02fQpkeXBsb3QucHJvcGhldChwcm9waGV0X2Z1bGwsIGZjc3Q9cHJlZGljdChwcm9waGV0X2Z1bGwsIHByb3BoZXRfZGYpKQpgYGAK