Machinelearningnotebooks: cannot import 'AutoMLConfig from azureml.train.automl
I tried to create a fresh environment using the three methods below. No matter what I can't import from azureml.train.automl. Help!
- using
automl/automl_env.ymlfrom this repo:conda create env -n AML_BTB -f automl_env.yml - The instructions given here.
- The instructions listed here:
pip install --upgrade azureml-sdk[notebooks,automl] - The instructions listed here
>>> from azureml.train.automl import AutoMLConfig
3.6.7 |Anaconda, Inc.| (default, Oct 28 2018, 19:44:12) [MSC v.1915 64 bit (AMD64)]
C:\Users\swanderz\Documents\attrition\aml_config\config.json
Found the config file in: C:\Users\swanderz\Documents\attrition\aml_config\config.json
avadevitsmlsvc westus2 RG-ITSMLTeam-Dev westus2
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Users\swanderz\AppData\Local\Continuum\anaconda3\envs\AML_BTB\lib\site-packages\azureml\train\automl\__init__.py", line 12, in <module>
from automl.client.core.common.utilities import extract_user_data, get_sdk_dependencies
File "C:\Users\swanderz\Documents\attrition\automl.py", line 40, in <module>
from azureml.train.automl import AutoMLConfig
ImportError: cannot import name 'AutoMLConfig'
swanderz
All 25 comments
sometimes i get this error.
should i be installing matplotlib before azureml-sdk? How about cython or numpy?
>>> import azureml.train.automl
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Users\swanderz\AppData\Local\Continuum\anaconda3\envs\AML1707\lib\site-packages\azureml\train\automl\__init__.py", line 12, in <module>
from automl.client.core.common.utilities import extract_user_data, get_sdk_dependencies
File "C:\Users\swanderz\Documents\attrition\automl.py", line 5, in <module>
import matplotlib
ModuleNotFoundError: No module named 'matplotlib'
matplotlib is included in the automl_env.yml. So, you may be missing this if you just did "pip install --upgrade azureml-sdk[notebooks,automl]" rather than "conda create env -n AML_BTB -f automl_env.yml.". matplotlib is not part of the core automl but is used in some of the sample notebooks.
In general I would recommend "conda create env -n AML_BTB -f automl_env.yml", which we also have wrapped by automl_setup.cmd. There is a README.md under https://github.com/Azure/MachineLearningNotebooks/tree/master/how-to-use-azureml/automated-machine-learning, which has more details.
Can you sent the output of "pip list" from the environment that you created (AML_BTB) ?
jeff-shepherd
on 11 Dec 2018
Hi jeff. Just tried again creating the environment from the automl_env.yml ( which i renamed to AMLjeff :) ),
Unfortunately, I still get an error from azureml.train.automl I've had this issue for the past 3 days and my coworker hasn't been able to help me. I have however, learned a great deal about Conda.
Here's my package list
Name Version Build Channel
_tflow_select 2.2.0 eigen
absl-py 0.6.1 py36_0
adal 1.2.0 <pip>
antlr4-python3-runtime 4.7.1 <pip>
applicationinsights 0.11.7 <pip>
argcomplete 1.9.4 <pip>
asn1crypto 0.24.0 py36_0
astor 0.7.1 py36_0
azure-cli-command-modules-nspkg 2.0.2 <pip>
azure-cli-core 2.0.52 <pip>
azure-cli-nspkg 3.0.3 <pip>
azure-cli-profile 2.1.2 <pip>
azure-cli-telemetry 1.0.0 <pip>
azure-common 1.1.16 <pip>
azure-graphrbac 0.53.0 <pip>
azure-mgmt-authorization 0.51.1 <pip>
azure-mgmt-containerregistry 2.5.0 <pip>
azure-mgmt-keyvault 1.1.0 <pip>
azure-mgmt-nspkg 3.0.2 <pip>
azure-mgmt-resource 2.0.0 <pip>
azure-mgmt-storage 3.1.0 <pip>
azure-ml-api-sdk 0.1.0a11 <pip>
azure-nspkg 3.0.2 <pip>
azure-storage-blob 1.4.0 <pip>
azure-storage-common 1.4.0 <pip>
azure-storage-nspkg 3.1.0 <pip>
azureml-core 1.0.2 <pip>
azureml-dataprep 0.5.3 <pip>
azureml-dataprep-native 11.1.2 <pip>
azureml-explain-model 1.0.2 <pip>
azureml-pipeline 1.0.2 <pip>
azureml-pipeline-core 1.0.2 <pip>
azureml-pipeline-steps 1.0.2 <pip>
azureml-sdk 1.0.2 <pip>
azureml-telemetry 1.0.2 <pip>
azureml-train 1.0.2 <pip>
azureml-train-automl 1.0.2 <pip>
azureml-train-core 1.0.2 <pip>
azureml-train-restclients-hyperdrive 1.0.2 <pip>
azureml-widgets 1.0.2 <pip>
azuremlftk 0.1.18323.5a1 <pip>
backcall 0.1.0 py36_0
backports.tempfile 1.0 <pip>
backports.weakref 1.0.post1 <pip>
bcrypt 3.1.4 <pip>
blas 1.0 mkl
bleach 3.0.2 py36_0
ca-certificates 2018.03.07 0
certifi 2018.11.29 py36_0
cffi 1.11.5 py36h74b6da3_1
chardet 3.0.4 <pip>
click 7.0 py36_0
cloudpickle 0.6.1 py36_0
colorama 0.4.0 py36_0
contextlib2 0.5.5 <pip>
cryptography 2.3.1 py36h74b6da3_0
cycler 0.10.0 py36h009560c_0
cython 0.29 py36ha925a31_0
cytoolz 0.9.0.1 py36hfa6e2cd_1
dask-core 1.0.0 py36_0
decorator 4.3.0 py36_0
dill 0.2.8.2 py36_0
distributed 1.23.1 <pip>
distributed 1.23.3 py36_0
distro 1.3.0 <pip>
docker 3.6.0 <pip>
docker-pycreds 0.4.0 <pip>
dotnetcore2 2.1.7 <pip>
entrypoints 0.2.3 py36_2
enum34 1.1.6 <pip>
freetype 2.8 h51f8f2c_1
gast 0.2.0 py36_0
grpcio 1.14.1 py36h5c4b210_0
h5py 2.8.0 py36h3bdd7fb_2
hdf5 1.10.2 hac2f561_1
heapdict 1.0.0 py36_2
humanfriendly 4.17 <pip>
icc_rt 2017.0.4 h97af966_0
icu 58.2 ha66f8fd_1
idna 2.7 py36_0
intel-openmp 2019.1 144
ipykernel 5.1.0 py36h39e3cac_0
ipython 7.2.0 py36h39e3cac_0
ipython_genutils 0.2.0 py36h3c5d0ee_0
ipywidgets 7.4.2 <pip>
isodate 0.6.0 <pip>
jedi 0.13.1 py36_0
jinja2 2.10 py36_0
jmespath 0.9.3 <pip>
joblib 0.13.0 <pip>
jpeg 9b hb83a4c4_2
JsonForm 0.0.2 <pip>
jsonpickle 1.0 <pip>
jsonschema 2.6.0 py36h7636477_0
JsonSir 0.0.2 <pip>
jupyter_client 5.2.3 py36_0
jupyter_core 4.4.0 py36_0
keras 2.2.4 0
keras-applications 1.0.6 py36_0
keras-base 2.2.4 py36_0
keras-preprocessing 1.0.5 py36_0
kiwisolver 1.0.1 <pip>
knack 0.5.1 <pip>
liac-arff 2.3.1 <pip>
libpng 1.6.35 h2a8f88b_0
libprotobuf 3.6.1 h7bd577a_0
libsodium 1.0.16 h9d3ae62_0
lightgbm 2.2.1 <pip>
m2w64-gcc-libgfortran 5.3.0 6
m2w64-gcc-libs 5.3.0 7
m2w64-gcc-libs-core 5.3.0 7
m2w64-gmp 6.1.0 2
m2w64-libwinpthread-git 5.0.0.4634.697f757 2
markdown 3.0.1 py36_0
markupsafe 1.1.0 py36he774522_0
matplotlib 2.1.0 py36h11b4b9c_0
matplotlib 3.0.2 <pip>
mistune 0.8.4 py36he774522_0
mkl 2018.0.3 1
mkl_fft 1.0.6 py36hdbbee80_0
mkl_random 1.0.1 py36h77b88f5_1
msgpack-python 0.5.6 py36he980bc4_1
msrest 0.6.2 <pip>
msrestazure 0.5.1 <pip>
msys2-conda-epoch 20160418 1
nb_conda 2.2.1 py36_0
nb_conda_kernels 2.2.0 py36_0
nbconvert 5.3.1 py36_0
nbformat 4.4.0 py36h3a5bc1b_0
ndg-httpsclient 0.5.1 <pip>
notebook 5.7.2 py36_0
numexpr 2.6.8 py36h9ef55f4_0
numpy 1.14.6 py36hc27ee41_4
numpy-base 1.14.6 py36h8128ebf_4
oauthlib 2.1.0 <pip>
openssl 1.0.2p hfa6e2cd_0
pandas 0.22.0 py36h6538335_0
pandas-ml 0.5.0 <pip>
pandoc 2.2.3.2 0
pandocfilters 1.4.2 py36_1
paramiko 2.4.2 <pip>
parso 0.3.1 py36_0
pathspec 0.5.9 <pip>
patsy 0.5.1 py36_0
pickleshare 0.7.5 py36_0
pip 18.1 py36_0
portalocker 1.2.1 <pip>
prometheus_client 0.4.2 py36_0
prompt_toolkit 2.0.7 py36_0
protobuf 3.6.1 py36h33f27b4_0
psutil 5.4.8 py36he774522_0
pyasn1 0.4.4 <pip>
pycparser 2.19 py36_0
pygments 2.2.0 py36hb010967_0
PyJWT 1.7.1 <pip>
PyNaCl 1.3.0 <pip>
pyodbc 4.0.24 py36h6538335_0
pyopenssl 18.0.0 py36_0
pyparsing 2.3.0 py36_0
pypiwin32 223 <pip>
pyqt 5.6.0 py36ha878b3d_6
pyreadline 2.1 py36_1
pysocks 1.6.8 py36_0
python 3.6.7 h33f27b4_1
python-dateutil 2.7.5 py36_0
Python-EasyConfig 0.1.7 <pip>
pytz 2018.7 py36_0
pywin32 224 <pip>
pywinpty 0.5.4 py36_0
pyyaml 3.13 py36hfa6e2cd_0
pyzmq 17.1.2 py36hfa6e2cd_0
qt 5.6.2 vc14h6f8c307_12
requests 2.21.0 <pip>
requests-oauthlib 1.0.0 <pip>
Resource 0.2.1 <pip>
ruamel.yaml 0.15.51 <pip>
scikit-learn 0.19.1 py36hae9bb9f_0
scipy 1.1.0 py36hc28095f_0
SecretStorage 2.3.1 <pip>
send2trash 1.5.0 py36_0
setuptools 40.6.2 py36_0
shap 0.25.2 <pip>
sip 4.18.1 py36h6538335_2
six 1.11.0 py36_1
sklearn-pandas 1.7.0 <pip>
sortedcontainers 2.1.0 py36_0
sqlite 3.25.3 he774522_0
statsmodels 0.9.0 py36h452e1ab_0
tabulate 0.8.2 <pip>
tblib 1.3.2 py36h30f5020_0
tensorboard 1.12.0 py36he025d50_0
tensorflow 1.12.0 eigen_py36h67ac661_0
tensorflow-base 1.12.0 eigen_py36h45df0d8_0
termcolor 1.1.0 py36_1
terminado 0.8.1 py36_1
testpath 0.4.2 py36_0
toolz 0.9.0 py36_0
tornado 5.1.1 py36hfa6e2cd_0
tqdm 4.28.1 <pip>
traitlets 4.3.2 py36h096827d_0
urllib3 1.23 py36_0
vc 14.1 h0510ff6_4
vs2015_runtime 14.15.26706 h3a45250_0
wcwidth 0.1.7 py36h3d5aa90_0
webencodings 0.5.1 py36_1
websocket-client 0.54.0 <pip>
werkzeug 0.14.1 py36_0
wheel 0.32.3 py36_0
widgetsnbextension 3.4.2 <pip>
win_inet_pton 1.0.1 py36_1
wincertstore 0.2 py36h7fe50ca_0
winpty 0.4.3 4
yaml 0.1.7 hc54c509_2
zeromq 4.2.5 he025d50_1
zict 0.1.3 py36_0
zlib 1.2.11 h62dcd97_3
More info:
After a lot of the messages like the following:
Requirement already satisfied: webencodings in c:\users\swanderz\appdata\local\continuum\anaconda3\envs\amljeff\lib\site-packages (from bleach->nbconvert->notebook>=4.4.1->widgetsnbextension~=3.4.0->ipywidgets>=7.0.0->azureml-widgets==1.0.2.*; extra == "notebooks"->azureml-sdk[automl,explain,notebooks]->-r C:\Users\swanderz\Documents\attrition\condaenv.xq0t5l_w.requirements.txt (line 2)) (0.5.1)
I get this
azure-cli-core 2.0.52 has requirement wheel==0.30.0, but you'll have wheel 0.32.3 which is incompatible.
azureml-train-automl 1.0.2 has requirement wheel==0.30.0, but you'll have wheel 0.32.3 which is incompatible.
Then a bunch of these
The script chardetect.exe is installed in 'C:\Users\swanderz\AppData\Local\Continuum\anaconda3\envs\AMLjeff\Scripts' which is not on PATH.
Consider adding this directory to PATH or, if you prefer to suppress this
warning, use --no-warn-script-location.
The wheel messages are normal and don't cause an error when running AutoML.
Also, the script not in the path warnings are normal. automl_setup.cmd suppresses them.
I don't see any issue with your pip list. The necessary libraries are present.
Can you make sure that this environment is active when you start jupyter notebook?
conda activate AMLjeff
Make sure that the correct kernel is active. This is selected with the Kernel | Change Kernel menu option.
If you still see the problem. Download the latest automl_env.yml and automl_setup.cmd for GitHub https://github.com/Azure/MachineLearningNotebooks/tree/master/how-to-use-azureml/automated-machine-learning and try updating conda with the command "conda update conda". Then try running the script automl_setup and pass a new environment name.
jeff-shepherd
on 11 Dec 2018
hey jeff. really appreciate your help here.
My troubleshooting instinct is to remove complexity so I've been troubleshooting with python 3 in the command line.
here's the steps i just took:
- downloaded the
automl_env.yml(i renamed it because you can't pass a name parameter to 'conda create env' when you're building from a yml file - ran
conda create env -f AMLjeff.yml - after install, `activate AMLjeff
- launch
python import azureml.train.automl-> ERROR
My theory is that maybe my conda installation is corrupted and I should start from scratch with that...
swanderz
on 11 Dec 2018
Thanks for the details.
I tried the sequence that you listed and it worked for me with a minor modification.
"conda create env" doesn't work on my system. I used "conda env create" instead (like in the automl_setup.cmd).
When I run python, it shows:
Python 3.6.7 |Anaconda, Inc.| (default, Dec 10 2018, 20:35:02) [MSC v.1915 64 bit (AMD64)] on win32
What do you see?
On your comment "(i renamed it because you can't pass a name parameter to 'conda create env' when you're building from a yml file", I can specify both -f and -n when using "conda env create". The -n overrides the "name:" value in the yml file.
If I don't specify -n, it uses the "name:" value in the yml file as the environment name. The name of the file doesn't matter.
Did you change the "name:" value inside AMLjeff.yml?
If you didn't, the environment name would be azure_automl rather than AMLjeff.
If you did, I agree that it looks like an issue with your conda installation. You could try upgrading with "conda update conda" or just installing again.
jeff-shepherd
on 11 Dec 2018
just now I've:
- completely uninstalled conda and reinstalled
- built the azure_automl env using
azure_automl.cmd - Got
from azureml.train.automl import AutoMLConfigto work.
I think I've isolated the issue to the VSCode Jupyter Extension. When the AML extension rebranded, I thought to try the Jupyter extension instead of AI tool's "open in jupyter server". The past two days has taught me otherwise!
I'm abandoning the idea of a REPL inside of VSCode and going back to jupyter as my IDE from here on out.
Thanks for all the support.
swanderz
on 11 Dec 2018
Thanks @swanderz! Your days of frustration that led you to your solution enabled me to solve the same problem in minutes.
DieselAnalytics
on 13 Feb 2019
@DieselAnalytics made my day thanks
swanderz
on 13 Feb 2019
@swanderz could you please point me to where i will find azure_automl.cmd file to setup the environment in conda
smfahad
on 10 Apr 2020
@smfahad setting up an azureml conda env can be tricky especially when automl is involved. Do you want to run the automl on your local machine or on an Azure ML compute target?
Check out this folder, and read the README. The .cmd file is there.
swanderz
on 10 Apr 2020
@swanderz I want to setup in my local machine. I trained a model using AzureML studio, now i want to run the model in my local machine. I completely uninstall and reinstall conda. Then i use automl_setup.cmd to setup a new environment as mentioned in README file but when i execute my code, i am still getting error.
ModuleNotFoundError: No module named 'azureml.train.automl.automlconfig'
I am wondering you mentioned azure_automl.cmd to setup new env whereas the available file is automl_setup.cmd as mentioned in README file. What am i missing?
smfahad
on 11 Apr 2020
@smfahad sorry that's a really crappy experience. everyone on my team has issues using the azureml-sdk with conda. i'd say once per month things break to the extent that I have to make a new local conda environement. and once a quarter, I have to uninstall and reinstall conda.
Unless @CESARDELATORRE says otherwise, I recommend you make a new environment and run
pip install azureml-sdk[automl,interpret]. This will install all the packages required for you to run automl runs (and hopefully use a pickled automl model) both on your local machine as well as submit runs remotely.
swanderz
on 11 Apr 2020
@swanderz I tried your suggestion even i completely remove anaconda and reinstall it from scratch. I then created a new environment and run pip install azureml-sdk[automl,interpret], but still I am getting the same error message "ModuleNotFoundError: No module named 'azureml.train.automl.automlconfig'" when running the model .
Any other advise or suggestion do you have?
smfahad
on 11 Apr 2020
Are other azureml modules importing correctly? Does the following work for you? You might not have your conda env properly registered as a Jupyter kernel.
from azureml.core import VERSION
print("Azure ML SDK Version: ", VERSION)
Yes, I recommend you make a new CONDA environment and run the pip installs stated here:
https://github.com/Azure/MachineLearningNotebooks/blob/master/NBSETUP.md
I've been doing that in several custom environments with no issues.
CESARDELATORRE
on 13 Apr 2020
@swanderz yes, it is working for me, i am getting the output "Azure ML SDK Version: 1.2.0"
@CESARDELATORRE I have setup a new environment as you suggested and tried again with the automl best deploy model downloaded from AzureML studio but still i am getting the same error.
ModuleNotFoundError Traceback (most recent call last)
3 import numpy as np
4 import pandas as pd
----> 5 import azureml.train.automl
6 from sklearn.externals import joblib
7 from azureml.core.model import Model
~\AppData\Roaming\Python\Python36\site-packagesazureml\trainautoml__init__.py in
26 # Suppress the warnings at the import phase.
27 warnings.simplefilter("ignore")
---> 28 from azureml.train.automl.automlconfig import AutoMLConfig
29
30 try:
ModuleNotFoundError: No module named 'azureml.train.automl.automlconfig'
smfahad
on 14 Apr 2020
can you run pip list | grep “azureml” in this environment and share the results?
swanderz
on 14 Apr 2020
@swanderz here is the output
azureml-automl-core 1.2.0
azureml-automl-runtime 1.2.0
azureml-contrib-notebook 1.2.0
azureml-contrib-pipeline-steps 1.2.0
azureml-contrib-server 1.2.0
azureml-contrib-services 1.2.0
azureml-core 1.2.0.post2
azureml-dataprep 1.3.6
azureml-dataprep-native 14.1.0
azureml-defaults 1.2.0
azureml-explain-model 1.2.0
azureml-interpret 1.2.0
azureml-model-management-sdk 1.0.1b6.post1
azureml-pipeline 1.2.0
azureml-pipeline-core 1.2.0
azureml-pipeline-steps 1.2.0
azureml-sdk 1.2.0
azureml-telemetry 1.2.0
azureml-tensorboard 1.2.0
azureml-train 1.2.0
azureml-train-automl 1.2.0
azureml-train-automl-client 1.2.0
azureml-train-automl-runtime 1.2.0
azureml-train-core 1.2.0
azureml-train-restclients-hyperdrive 1.2.0
azureml-widgets 1.2.0
smfahad
on 14 Apr 2020
Also to be clear, what’s your import statement? Below is what the reference tutorial suggests. I recommend using that as it is more likely to stay Up to date compared to the notebooks in this repo.
from azureml.train.automl import AutoMLConfig
swanderz
on 14 Apr 2020
@swanderz Even when i am just running the mention statement without any other statement in my notebook, i am getting the error.
from azureml.train.automl import AutoMLConfig
ModuleNotFoundError Traceback (most recent call last)
----> 1 from azureml.train.automl import AutoMLConfig
~\AppData\Roaming\Python\Python36\site-packagesazureml\trainautoml__init__.py in
26 # Suppress the warnings at the import phase.
27 warnings.simplefilter("ignore")
---> 28 from azureml.train.automl.automlconfig import AutoMLConfig
29
30 try:
ModuleNotFoundError: No module named 'azureml.train.automl.automlconfig'
smfahad
on 14 Apr 2020
I'm sorry this is so difficult, I can definitely empathize. I'm 85% sure that the issue has to do with your jupyter notebook is not using your conda environment.
Fix
Open the anaconda prompt and try this sequence of commands.
(base) C:\Users\anders.swanson>conda activate ret
(ret) C:\Users\anders.swanson>python
Python 3.6.2 |Anaconda, Inc.| (default, Sep 30 2017, 11:52:29) [MSC v.1900 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from azureml.core import VERSION
>>> print("Azure ML SDK Version: ", VERSION)
Azure ML SDK Version: 1.3.0
>>> from azureml.train.automl import AutoMLConfig
>>>
If the above works, I recommend that you read this article about setting up an env on you local computer. Specifically the part about registering a new kernel w/ the ipykernel package.
Root Cause
Look at the path of your stacktrace.
~\AppData\Roaming\Python\Python36\site-packages\azureml\train\automl_init_.py
The stacktrace should say something like
~\AppData\Local\Continuum\anaconda3\envs\{YOUR_ENV_NAME}\lib\site-packages\azureml\train\automl\__init__.py"
My guess is that you installed the Azure ML SDK into your system (i.e. non-anaconda) Python a long time ago. So all your conda work is valid, but jupyter is using a kernel pointing to the wrong place.
swanderz
on 14 Apr 2020
My guess is that you installed the Azure ML SDK into your system (i.e. non-anaconda) Python a long time ago. So all your conda work is valid, but jupyter is using a kernel pointing to the wrong place.
@smfahad I usually face something similar with some packages, whenever in doubt, I check where these packages are being summoned from using .__file__ (if they have that implemented).
For azureml's case, you would try:
import azureml.core as az
az.__file__ #check from where it's being loaded
import azureml.train as azt
azt.__file__ # check from where it's being loaded
jadhosn
on 14 Apr 2020
@swanderz Thanks for your help. Very much appreciated to identify the problem and taking it to the right direction.
When i run your mentioned commands within conda activated environment, the azureml.core package imported correctly and pointing to the package with in my environment as @jadhosn mentioned, which seems fine.
But when i imported azureml.train.automl, it through an exception pointing the package to the global package directory (~\AppData\Roaming\Python\Python36\site-packagesazureml\trainautoml_init_.py). I am not sure why this happens as one package is being imported from my environment package directory and the other is from the global package directory.
So I did the following:
1- Uninstall both anaconda and python completely.
2- Deleted all the folders related to anaconda from C:\Users{username} (like .anaconda, .conda)
3- Deleted the complete folder c:\users{username}appdata\roamingpython\
Then reinstalled Anaconda, created a new environment and install all the pip packages as mentioned by @CESARDELATORRE
and there you go, it is now working fine.
Thanks again for your kind support. @swanderz @CESARDELATORRE @jadhosn
smfahad
on 14 Apr 2020
Cool! I'm glad it worked for you! đź‘Ť
CESARDELATORRE
on 15 Apr 2020
Related issues
swanderz
·
5Comments
tkawchak
·
5Comments
nswitanek
·
4Comments
ashic
·
3Comments
casieo
·
4Comments
Most helpful comment
Thanks @swanderz! Your days of frustration that led you to your solution enabled me to solve the same problem in minutes.