esp32-s3 esp-idf linux下开发环境搭建流程

说明

最近买了块 esp32-s3 的最小开发板，YD-esp32。本想着开发环境跟着官方文档走应该很愉快，没想到一波三折，在这里记录一下。

我是在 Ubuntu18.04 上搭环境，如果你是其他系统，可能会碰到其他问题，或者不会碰到我遇到的问题。

下载 esp-idf 源码

首先第一步下载源码就出了问题，下载不成功。我参考的官方文档链接如下
https://docs.espressif.com/projects/esp-idf/zh_CN/latest/esp32/get-started/linux-macos-setup.html

推荐下载命令是 git clone --recursive https://github.com/espressif/esp-idf.git 由于大家都知道的原因，国内的下载不方便。但是这个文档没提到这一点，我到网上查才知道乐鑫有在 gitee 放了源码，也作了说明，链接如下
https://gitee.com/EspressifSystems/esp-gitee-tools/blob/master/docs/README-submodule-update.md

可以通过以下方式下载源码：
下载两个仓库

git clone https://gitee.com/EspressifSystems/esp-gitee-tools.git
git clone https://gitee.com/EspressifSystems/esp-idf.git

进入 esp-idf 目录，利用 esp-gitee-tools 目录中的脚本来更新 submodules

cd esp-idf
../esp-gitee-tools/submodule-update.sh

到这里源码下载完成

配置编译 esp-idf 所需的环境

主要的坑都在这里，这一步很依赖 python 环境，所以如果对应的 python 版本不符合要求，就会出现比较多的错误。
1.首先按照步骤执行 ./install.sh esp32s3 (因为我的板子是 esp32-s3 所以这里的目标芯片就指定为 esp32s3)，报如下错误

liu@liu-dev:~/work/projects/esp32_s3/esp-idf$ ./install.sh esp32s3
Detecting the Python interpreter
Checking "python3" ...
Checking "python" ...
"python" has been detected
Checking Python compatibility
Traceback (most recent call last):
  File "/home/liu/work/projects/esp32_s3/esp-idf/tools/python_version_checker.py", line 45, in <module>
    check()
  File "/home/liu/work/projects/esp32_s3/esp-idf/tools/python_version_checker.py", line 39, in check
    _ver_to_str(OLDEST_PYTHON_SUPPORTED), _ver_to_str(sys.version_info[:3])
RuntimeError: ESP-IDF supports Python 3.7 or newer but you are using Python 2.7.17. Please upgrade your installation as described in the documentation.

上述信息提示要 Python 3.7 以上，我用的是 ubuntu18.04 ，通过执行 python3 --version，得知当前系统用的是 Python 3.6.9，不符合要求。
安装 Python 3.7

sudo apt install python3.7-minimal

2.接着继续上面 ./install.sh esp32s3，出现以下错误

Extracting /home/liu/.espressif/dist/openocd-esp32-linux-amd64-0.11.0-esp32-20220411.tar.gz to /home/liu/.espressif/tools/openocd-esp32/v0.11.0-esp32-20220411
Installing Python environment and packages
Creating a new Python environment in /home/liu/.espressif/python_env/idf5.0_py3.7_env
The virtual environment was not created successfully because ensurepip is not
available.  On Debian/Ubuntu systems, you need to install the python3-venv
package using the following command.

    apt-get install python3-venv

You may need to use sudo with that command.  After installing the python3-venv
package, recreate your virtual environment.

Failing command: ['/home/liu/.espressif/python_env/idf5.0_py3.7_env/bin/python', '-Im', 'ensurepip', '--upgrade', '--default-pip']

Traceback (most recent call last):
  File "/home/liu/work/projects/esp32_s3/esp-idf/tools/idf_tools.py", line 2497, in <module>
    main(sys.argv[1:])
  File "/home/liu/work/projects/esp32_s3/esp-idf/tools/idf_tools.py", line 2489, in main
    action_func(args)
  File "/home/liu/work/projects/esp32_s3/esp-idf/tools/idf_tools.py", line 1978, in action_install_python_env
    stdout=sys.stdout, stderr=sys.stderr)
  File "/usr/lib/python3.7/subprocess.py", line 363, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/usr/bin/python', '-m', 'venv', '--clear', '/home/liu/.espressif/python_env/idf5.0_py3.7_env']' returned non-zero exit status 1.

到网上一查说是什么配置虚拟环境有问题，一开始按照信息提示，apt-get install python3-venv，安装完仍然有上述错误。参考了别人的文档，要安装对应的版本。步骤如下

sudo apt-get install python3.7-venv
rm -rf /home/liu/.espressif/python_env/idf5.0_py3.7_env #删除原来的环境
find . -name "*.pyc" | xargs rm -f #同上

3.继续执行 ./install.sh esp32s3，还是报错了

liu@liu-dev:~/work/projects/esp32_s3/esp-idf$ ./install.sh esp32s3
Detecting the Python interpreter
Checking "python3" ...
Python 3.7.5
"python3" has been detected
Checking Python compatibility
Installing ESP-IDF tools
Current system platform: linux-amd64
Selected targets are: esp32s3
Installing tools: xtensa-esp-elf-gdb, xtensa-esp32s3-elf, riscv32-esp-elf, esp32s2ulp-elf, openocd-esp32
Skipping xtensa-esp-elf-gdb@11.2_20220529 (already installed)
Skipping xtensa-esp32s3-elf@esp-2022r1-RC1-11.2.0 (already installed)
Skipping riscv32-esp-elf@esp-2022r1-RC1-11.2.0 (already installed)
Skipping esp32s2ulp-elf@2.28.51-esp-20191205 (already installed)
Skipping openocd-esp32@v0.11.0-esp32-20220411 (already installed)
Installing Python environment and packages
Python 3.7.5
pip 9.0.1 from /home/liu/.espressif/python_env/idf5.0_py3.7_env/lib/python3.7/site-packages (python 3.7)
Skipping the download of /home/liu/.espressif/espidf.constraints.v5.0.txt because it was downloaded recently. If you believe that this is causing you trouble then remove it manually and re-run your install script.
Installing Python packages
 Constraint file: /home/liu/.espressif/espidf.constraints.v5.0.txt
 Requirement files:
  - /home/liu/work/projects/esp32_s3/esp-idf/tools/requirements/requirements.core.txt

Usage:
  /home/liu/.espressif/python_env/idf5.0_py3.7_env/bin/python -m pip install [options] <requirement specifier> [package-index-options] ...
  /home/liu/.espressif/python_env/idf5.0_py3.7_env/bin/python -m pip install [options] -r <requirements file> [package-index-options] ...
  /home/liu/.espressif/python_env/idf5.0_py3.7_env/bin/python -m pip install [options] [-e] <vcs project url> ...
  /home/liu/.espressif/python_env/idf5.0_py3.7_env/bin/python -m pip install [options] [-e] <local project path> ...
  /home/liu/.espressif/python_env/idf5.0_py3.7_env/bin/python -m pip install [options] <archive url/path> ...

no such option: --no-warn-script-location
Traceback (most recent call last):
  File "/home/liu/work/projects/esp32_s3/esp-idf/tools/idf_tools.py", line 2497, in <module>
    main(sys.argv[1:])
  File "/home/liu/work/projects/esp32_s3/esp-idf/tools/idf_tools.py", line 2489, in main
    action_func(args)
  File "/home/liu/work/projects/esp32_s3/esp-idf/tools/idf_tools.py", line 2010, in action_install_python_env
    subprocess.check_call(run_args, stdout=sys.stdout, stderr=sys.stderr, env=env_copy)
  File "/usr/lib/python3.7/subprocess.py", line 363, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/home/liu/.espressif/python_env/idf5.0_py3.7_env/bin/python', '-m', 'pip', 'install', '--no-warn-script-location', '-r', '/home/liu/work/projects/esp32_s3/esp-idf/tools/requirements/requirements.core.txt', '--upgrade', '--constraint', '/home/liu/.espressif/espidf.constraints.v5.0.txt', '--extra-index-url', 'https://dl.espressif.com/pypi']' returned non-zero exit status 2.

关键的信息是 no such option: --no-warn-script-location，到网上一查说删掉这个选项就行，因为错误信息最底下有提示是执行什么 command 时出错
所以执行以下命令

/home/liu/.espressif/python_env/idf5.0_py3.7_env/bin/python -m pip install -r /home/liu/work/projects/esp32_s3/esp-idf/tools/requirements/requirements.core.txt --upgrade --constraint /home/liu/.espressif/espidf.constraints.v5.0.txt --extra-index-url https://dl.espressif.com/pypi

然后还是报错了。

Building wheels for collected packages: cffi
  Running setup.py bdist_wheel for cffi ... error
  Complete output from command /home/liu/.espressif/python_env/idf5.0_py3.7_env/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-tfo1_8i4/cffi/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/tmp_pir97mxpip-wheel- --python-tag cp37:
  usage: -c [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
     or: -c --help [cmd1 cmd2 ...]
     or: -c --help-commands
     or: -c cmd --help

  error: invalid command 'bdist_wheel'

  ----------------------------------------
  Failed building wheel for cffi
  Running setup.py clean for cffi
Failed to build cffi
Installing collected packages: setuptools, typing-extensions, zipp, importlib-metadata, click, pyserial, future, pycparser, cffi, six, cryptography, pyparsing, pyelftools, contextlib2, schema, charset-normalizer, certifi, idna, urllib3, requests, pyyaml, tqdm, requests-toolbelt, colorama, idf-component-manager, bitstring, ecdsa, reedsolo, esptool, construct, pygdbmi, esp-coredump, kconfiglib, freertos-gdb
  Found existing installation: setuptools 39.0.1
    Uninstalling setuptools-39.0.1:
      Successfully uninstalled setuptools-39.0.1
  Running setup.py install for cffi ... error
    Complete output from command /home/liu/.espressif/python_env/idf5.0_py3.7_env/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-tfo1_8i4/cffi/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-vvl_mqyx-record/install-record.txt --single-version-externally-managed --compile --install-headers /home/liu/.espressif/python_env/idf5.0_py3.7_env/include/site/python3.7/cffi:
    /home/liu/.espressif/python_env/idf5.0_py3.7_env/lib/python3.7/site-packages/setuptools/config/setupcfg.py:463: SetuptoolsDeprecationWarning: The license_file parameter is deprecated, use license_files instead.
      warnings.warn(msg, warning_class)
    running install
    /home/liu/.espressif/python_env/idf5.0_py3.7_env/lib/python3.7/site-packages/setuptools/command/install.py:37: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
      setuptools.SetuptoolsDeprecationWarning,
    running build
    running build_py
    creating build
    creating build/lib.linux-x86_64-cpython-37
    creating build/lib.linux-x86_64-cpython-37/cffi
    copying cffi/pkgconfig.py -> build/lib.linux-x86_64-cpython-37/cffi
    copying cffi/setuptools_ext.py -> build/lib.linux-x86_64-cpython-37/cffi
    copying cffi/model.py -> build/lib.linux-x86_64-cpython-37/cffi
    copying cffi/ffiplatform.py -> build/lib.linux-x86_64-cpython-37/cffi
    copying cffi/commontypes.py -> build/lib.linux-x86_64-cpython-37/cffi
    copying cffi/api.py -> build/lib.linux-x86_64-cpython-37/cffi
    copying cffi/vengine_gen.py -> build/lib.linux-x86_64-cpython-37/cffi
    copying cffi/cparser.py -> build/lib.linux-x86_64-cpython-37/cffi
    copying cffi/cffi_opcode.py -> build/lib.linux-x86_64-cpython-37/cffi
    copying cffi/recompiler.py -> build/lib.linux-x86_64-cpython-37/cffi
    copying cffi/lock.py -> build/lib.linux-x86_64-cpython-37/cffi
    copying cffi/error.py -> build/lib.linux-x86_64-cpython-37/cffi
    copying cffi/verifier.py -> build/lib.linux-x86_64-cpython-37/cffi
    copying cffi/__init__.py -> build/lib.linux-x86_64-cpython-37/cffi
    copying cffi/backend_ctypes.py -> build/lib.linux-x86_64-cpython-37/cffi
    copying cffi/vengine_cpy.py -> build/lib.linux-x86_64-cpython-37/cffi
    copying cffi/_cffi_include.h -> build/lib.linux-x86_64-cpython-37/cffi
    copying cffi/parse_c_type.h -> build/lib.linux-x86_64-cpython-37/cffi
    copying cffi/_embedding.h -> build/lib.linux-x86_64-cpython-37/cffi
    copying cffi/_cffi_errors.h -> build/lib.linux-x86_64-cpython-37/cffi
    running build_ext
    building '_cffi_backend' extension
    creating build/temp.linux-x86_64-cpython-37
    creating build/temp.linux-x86_64-cpython-37/c
    x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -DFFI_BUILDING=1 -DUSE__THREAD -DHAVE_SYNC_SYNCHRONIZE -I/home/liu/.espressif/python_env/idf5.0_py3.7_env/include -I/usr/include/python3.7m -c c/_cffi_backend.c -o build/temp.linux-x86_64-cpython-37/c/_cffi_backend.o
    c/_cffi_backend.c:2:10: fatal error: Python.h: 没有那个文件或目录
     #include <Python.h>
              ^~~~~~~~~~
    compilation terminated.
    error: command '/usr/bin/x86_64-linux-gnu-gcc' failed with exit code 1

    ----------------------------------------
Command "/home/liu/.espressif/python_env/idf5.0_py3.7_env/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-tfo1_8i4/cffi/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-vvl_mqyx-record/install-record.txt --single-version-externally-managed --compile --install-headers /home/liu/.espressif/python_env/idf5.0_py3.7_env/include/site/python3.7/cffi" failed with error code 1 in /tmp/pip-build-tfo1_8i4/cffi/

从上面看出找不到头文件 Python.h，执行以下命令解决

sudo apt-get install python3.7-dev

再执行 /home/liu/.espressif/python_env/idf5.0_py3.7_env/bin/python -m pip install -r /home/liu/work/projects/esp32_s3/esp-idf/tools/requirements/requirements.core.txt --upgrade --constraint /home/liu/.espressif/espidf.constraints.v5.0.txt --extra-index-url https://dl.espressif.com/pypi
提示以下错误

Building wheels for collected packages: cffi
  Running setup.py bdist_wheel for cffi ... error
  Complete output from command /home/liu/.espressif/python_env/idf5.0_py3.7_env/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-6kahl95f/cffi/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/tmpfntsyykipip-wheel- --python-tag cp37:
  /home/liu/.espressif/python_env/idf5.0_py3.7_env/lib/python3.7/site-packages/setuptools/config/setupcfg.py:463: SetuptoolsDeprecationWarning: The license_file parameter is deprecated, use license_files instead.
    warnings.warn(msg, warning_class)
  usage: -c [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
     or: -c --help [cmd1 cmd2 ...]
     or: -c --help-commands
     or: -c cmd --help

  error: invalid command 'bdist_wheel'

  ----------------------------------------
  Failed building wheel for cffi
  Running setup.py clean for cffi
Failed to build cffi
Installing collected packages: six, cffi, cryptography, pyparsing, pyelftools, contextlib2, schema, urllib3, charset-normalizer, certifi, idna, requests, colorama, pyyaml, tqdm, requests-toolbelt, idf-component-manager, bitstring, reedsolo, ecdsa, esptool, pygdbmi, construct, esp-coredump, kconfiglib, freertos-gdb
Exception:
Traceback (most recent call last):
  File "/home/liu/.espressif/python_env/idf5.0_py3.7_env/lib/python3.7/site-packages/pip/basecommand.py", line 215, in main
    status = self.run(options, args)
  File "/home/liu/.espressif/python_env/idf5.0_py3.7_env/lib/python3.7/site-packages/pip/commands/install.py", line 360, in run
    prefix=options.prefix_path,
  File "/home/liu/.espressif/python_env/idf5.0_py3.7_env/lib/python3.7/site-packages/pip/req/req_set.py", line 784, in install
    **kwargs
  File "/home/liu/.espressif/python_env/idf5.0_py3.7_env/lib/python3.7/site-packages/pip/req/req_install.py", line 851, in install
    self.move_wheel_files(self.source_dir, root=root, prefix=prefix)
  File "/home/liu/.espressif/python_env/idf5.0_py3.7_env/lib/python3.7/site-packages/pip/req/req_install.py", line 1064, in move_wheel_files
    isolated=self.isolated,
  File "/home/liu/.espressif/python_env/idf5.0_py3.7_env/lib/python3.7/site-packages/pip/wheel.py", line 247, in move_wheel_files
    prefix=prefix,
  File "/home/liu/.espressif/python_env/idf5.0_py3.7_env/lib/python3.7/site-packages/pip/locations.py", line 153, in distutils_scheme
    i.finalize_options()
  File "/home/liu/.espressif/python_env/idf5.0_py3.7_env/share/python-wheels/setuptools-39.0.1-py2.py3-none-any.whl/setuptools/command/install.py", line 38, in finalize_options
    orig.install.finalize_options(self)
  File "/usr/lib/python3.7/distutils/command/install.py", line 390, in finalize_options
    ('build_lib', 'build_lib'))
  File "/usr/lib/python3.7/distutils/cmd.py", line 286, in set_undefined_options
    src_cmd_obj = self.distribution.get_command_obj(src_cmd)
  File "/usr/lib/python3.7/distutils/dist.py", line 857, in get_command_obj
    klass = self.get_command_class(command)
  File "/home/liu/.espressif/python_env/idf5.0_py3.7_env/share/python-wheels/setuptools-39.0.1-py2.py3-none-any.whl/setuptools/dist.py", line 634, in get_command_class
    self.cmdclass[command] = cmdclass = ep.load()
  File "/home/liu/.espressif/python_env/idf5.0_py3.7_env/share/python-wheels/pkg_resources-0.0.0-py2.py3-none-any.whl/pkg_resources/__init__.py", line 2324, in load
    return self.resolve()
  File "/home/liu/.espressif/python_env/idf5.0_py3.7_env/share/python-wheels/pkg_resources-0.0.0-py2.py3-none-any.whl/pkg_resources/__init__.py", line 2330, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
ModuleNotFoundError: No module named 'setuptools.command.build'

提示 No module named ‘setuptools.command.build’
找到以下参考的步骤
https://stackoverflow.com/questions/14426491/python-3-importerror-no-module-named-setuptools
我执行了以下命令才不会有上述错误

sudo apt-get install python3.7-setuptools
pip3 install setuptools
pip3 install -U pip

到这里官方文档提到 ./install.sh esp32s3 才成功了
4.解决掉 install.sh 出现的错误之后，执行 source export.sh 才没有问题。到这里基本环境就搭好了。

编译下载运行 helloword 工程。

1.回到 esp-idf 的上一级目录，拷贝 esp-idf自带的 helloword工程

cp -r esp-idf/examples/get-started/hello_world .

然后编译的时候出错了

liu@liu-dev:~/work/projects/esp32_s3/hello_world$ idf.py build
Executing action: all (aliases: build)
Running cmake in directory /home/liu/work/projects/esp32_s3/hello_world/build
Executing "cmake -G Ninja -DPYTHON_DEPS_CHECKED=1 -DESP_PLATFORM=1 -DCCACHE_ENABLE=0 /home/liu/work/projects/esp32_s3/hello_world"...
CMake Error at CMakeLists.txt:3 (cmake_minimum_required):
  CMake 3.16 or higher is required.  You are running version 3.10.2


-- Configuring incomplete, errors occurred!
cmake failed with exit code 1

cmake版本不对，要 3.16 以上，到 cmake 官网下一个
https://github.com/Kitware/CMake/releases
我下的是这个版本 cmake-3.16.8-Linux-x86_64.tar.gz
执行以下命令解压

tar -xzf cmake-3.16.8-Linux-x86_64.tar.gz

配置cmake到环境变量

 vi ~/.bashrc
 export PATH=/home/liu/bin/cmake-3.16.8-Linux-x86_64/bin:$PATH

上述目录取决于解压的cmake位置
执行以下命令，终于编译成功

idf.py set-target esp32s3
idf.py build

2.下载编译好的二进制文件到开发板
我用的串口下载，用电脑连接好开发板后，可以发现多了个 /dev/ttyACM0，这就是要操作的端口
执行以下命令下载

ipf.py -p /dev/ttyACM0 flash

一开始提示操作 /dev/ttyACM0 没权限，执行

sudo chmod 777 /dev/ttyACM0

再执行 ipf.py -p /dev/ttyACM0 flash 就可以了，然后用串口工具就可以看到 helloword 的打印了

至此环境搭建完毕