nanopb - Protocol Buffers 编译

关于Protocol Buffers

Protocol Buffers是Google开发的用于序列化数据的机制，它无关开发语言，无关使用平台，且具有良好的可扩展性。
使用Protocol Buffers，你只需要把你的数据定义为结构化的protocol buffer数据，然后使用编译工具把它编译成特定语言下的代码，调用编译后的代码接口，你就能很方便地将你的结构数据写成数据流（编码），或是将数据流读取为你的结构数据(解码)。
目前Protocol buffers可支持的生成语言有Java,Python, Object-c, 以及C++, proto3版本还新增了Dart,Go, Ruby, C#。

编译工具安装

这里说的编译工具是指编译.proto文件，即protocol compiler(protoc)
首先，安装并选择默认python版本：

sudo apt-get install python//系统自带的python脚本，18.04上自带版本仍然是2.7.15~rc1-1)
//https://www.runoob.com/python3/python3-install.html　安装python3
//使用update-alternatives修改系统默认配置python使用python3

update-alternatives --list python//列出所有可用的python版本
	#/usr/bin/python2.7
	#/usr/bin/python3.6
sudo update-alternatives --install /usr/bin/python python /usr/bin/python2.7 1
sudo update-alternatives --install /usr/bin/python python /usr/bin/python3.6 2
//将python3.6优先级设为２，就会自动设置为默认python版本
python --version
	#Python 3.6.9

update-alternatives --config python//也可以通过此命令修改默认python版本
	有 2 个候选项可用于替换 python (提供 /usr/bin/python)。

  选择       路径              优先级  状态
------------------------------------------------------------
* 0            /usr/bin/python3.6   2         自动模式
  1            /usr/bin/python2.7   1         手动模式
  2            /usr/bin/python3.6   2         手动模式

要维持当前值[*]请按<回车键>，或者键入选择的编号：

再者，有了python，通过pip来安装protobuf,protoc

sudo apt install python-pip//如未安装，先安装pip
pip install protobuf grpcio-tools

protoc是用C++语言编写的，如果你使用的是C++语言,可以参考C++安装说明来安装protoc
非C++语言也可以参考protobuf来安装

Nanopb

Nanopb is a small code-size Protocol Buffers implementation in ansi C. It is especially suitable for use in microcontrollers, but fits any memory restricted system.

使用nanopb库编译.proto文件和工程使用，只需要下面两步即可：

1. 使用protoc编译.protoc文件；
2. 在你的工程中import pb_encode.c, pb_decode.c 和 pb_common.c

demo具体参考nanopb中example/simple工程，编译.proto文件的命令可参考如下：

protomk:
	mkdir -p $(OUT_DIR)
	$(PROTOC) $(PROTOC_OPTS) --proto_path=$(PROTO_DIR) --nanopb_out=$(OUT_DIR) $(PROTO_DIR)/*.proto
//OUT_DIR是%.proto文件编译成功后,%.pb.c %.pb.h输出的文件目录
//PROTO_DIR是.proto文件所在的目录位置
//PROTOC和PROTOC_OPTS的定义在../../extra/nanopb.mk

//extra/nanopb.mk
ifneq "$(wildcard $(NANOPB_DIR)/generator-bin)" ""
	# Binary package
	PROTOC = $(NANOPB_DIR)/generator-bin/protoc
	PROTOC_OPTS = 
else
	# Source only or git checkout
	PROTOC = protoc
	ifdef WINDOWS
		PROTOC_OPTS = --plugin=protoc-gen-nanopb=$(NANOPB_DIR)/generator/protoc-gen-nanopb.bat
	else
		PROTOC_OPTS = --plugin=protoc-gen-nanopb=$(NANOPB_DIR)/generator/protoc-gen-nanopb
	endif
endif

我自己的工程在ubuntu18.04上此命令打印出来如下：

protoc --plugin=protoc-gen-nanopb=/media/pc-name/nanopb-nanopb-0.3.9.2/generator/protoc-gen-nanopb --proto_path=./proto --nanopb_out=./out ./proto/*.proto

遇到的问题

问题1: grpcio-tools下载超时中断

xiaoyanxia@xiaoyanxia-pc:/usr/bin$ pip3 install grpcio-tools
Collecting grpcio-tools
  Using cached https://files.pythonhosted.org/packages/7a/b3/532a19e668920d3e1c1c832b4365734ffe6f510c9dd5e1c81240d82535ea/grpcio-tools-1.31.0.tar.gz
Collecting protobuf<4.0dev,>=3.5.0.post1 (from grpcio-tools)
  Using cached https://files.pythonhosted.org/packages/30/79/510974552cebff2ba04038544799450defe75e96ea5f1675dbf72cc8744f/protobuf-3.13.0-cp36-cp36m-manylinux1_x86_64.whl
Collecting grpcio>=1.31.0 (from grpcio-tools)
  Downloading https://files.pythonhosted.org/packages/e3/0e/f56aa1f8200ae3c5d38305e69f5920caa480c7ff54ae4d8a5f57d1d69fa4/grpcio-1.31.0.tar.gz (20.0MB)
    1% |▌                               | 327kB 14kB/s eta 0:22:34^C
Operation cancelled by user
xiaoyanxia@xiaoyanxia-pc:/usr/bin$ pip3 install grpcio
Collecting grpcio
  Downloading https://files.pythonhosted.org/packages/e3/0e/f56aa1f8200ae3c5d38305e69f5920caa480c7ff54ae4d8a5f57d1d69fa4/grpcio-1.31.0.tar.gz (20.0MB)
    4% |█▌                              | 931kB 9.1kB/s eta 0:35:04Exception:
Traceback (most recent call last):
  File "/usr/share/python-wheels/urllib3-1.22-py2.py3-none-any.whl/urllib3/response.py", line 302, in _error_catcher
    yield
  File "/usr/share/python-wheels/urllib3-1.22-py2.py3-none-any.whl/urllib3/response.py", line 384, in read
    data = self._fp.read(amt)
  File "/usr/share/python-wheels/CacheControl-0.11.7-py2.py3-none-any.whl/cachecontrol/filewrapper.py", line 60, in read
    data = self.__fp.read(amt)
  File "/usr/lib/python3.6/http/client.py", line 459, in read
    n = self.readinto(b)
  File "/usr/lib/python3.6/http/client.py", line 503, in readinto
    n = self.fp.readinto(b)
  File "/usr/lib/python3.6/socket.py", line 586, in readinto
    return self._sock.recv_into(b)
  File "/usr/lib/python3.6/ssl.py", line 1012, in recv_into
    return self.read(nbytes, buffer)
  File "/usr/lib/python3.6/ssl.py", line 874, in read
    return self._sslobj.read(len, buffer)
  File "/usr/lib/python3.6/ssl.py", line 631, in read
    v = self._sslobj.read(len, buffer)
socket.timeout: The read operation timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/pip/basecommand.py", line 215, in main
    status = self.run(options, args)
  File "/usr/lib/python3/dist-packages/pip/commands/install.py", line 353, in run
    wb.build(autobuilding=True)
  File "/usr/lib/python3/dist-packages/pip/wheel.py", line 749, in build
    self.requirement_set.prepare_files(self.finder)
  File "/usr/lib/python3/dist-packages/pip/req/req_set.py", line 380, in prepare_files
    ignore_dependencies=self.ignore_dependencies))
  File "/usr/lib/python3/dist-packages/pip/req/req_set.py", line 620, in _prepare_file
    session=self.session, hashes=hashes)
  File "/usr/lib/python3/dist-packages/pip/download.py", line 821, in unpack_url
    hashes=hashes
  File "/usr/lib/python3/dist-packages/pip/download.py", line 659, in unpack_http_url
    hashes)
  File "/usr/lib/python3/dist-packages/pip/download.py", line 882, in _download_http_url
    _download_url(resp, link, content_file, hashes)
  File "/usr/lib/python3/dist-packages/pip/download.py", line 603, in _download_url
    hashes.check_against_chunks(downloaded_chunks)
  File "/usr/lib/python3/dist-packages/pip/utils/hashes.py", line 46, in check_against_chunks
    for chunk in chunks:
  File "/usr/lib/python3/dist-packages/pip/download.py", line 571, in written_chunks
    for chunk in chunks:
  File "/usr/lib/python3/dist-packages/pip/utils/ui.py", line 139, in iter
    for x in it:
  File "/usr/lib/python3/dist-packages/pip/download.py", line 560, in resp_read
    decode_content=False):
  File "/usr/share/python-wheels/urllib3-1.22-py2.py3-none-any.whl/urllib3/response.py", line 436, in stream
    data = self.read(amt=amt, decode_content=decode_content)
  File "/usr/share/python-wheels/urllib3-1.22-py2.py3-none-any.whl/urllib3/response.py", line 401, in read
    raise IncompleteRead(self._fp_bytes_read, self.length_remaining)
  File "/usr/lib/python3.6/contextlib.py", line 99, in __exit__
    self.gen.throw(type, value, traceback)
  File "/usr/share/python-wheels/urllib3-1.22-py2.py3-none-any.whl/urllib3/response.py", line 307, in _error_catcher
    raise ReadTimeoutError(self._pool, None, 'Read timed out.')
urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='files.pythonhosted.org', port=443): Read timed out.

解决方案：重新打开一个新的终端重新下载，就是这么莫名奇妙，好几次都是如此解决

问题2: make失败1

Traceback (most recent call last):
  File "../../generator/nanopb_generator.py", line 2123, in <module>
    main_cli()
  File "../../generator/nanopb_generator.py", line 2019, in main_cli
    results = process_file(fdesc.name, fdesc, options, other_files)
  File "../../generator/nanopb_generator.py", line 1954, in process_file
    headerdata = ''.join(f.generate_header(includes, headerbasename, options))
  File "../../generator/nanopb_generator.py", line 1610, in generate_header
    yield msg.fields_declaration(self.dependencies) + '\n'
  File "../../generator/nanopb_generator.py", line 1169, in fields_declaration
    defval = self.default_value(dependencies)
  File "../../generator/nanopb_generator.py", line 1299, in default_value
    desc = google.protobuf.descriptor.MakeDescriptor(optional_only)
  File "/home/pc-name/.local/lib/python3.6/site-packages/google/protobuf/descriptor.py", line 1077, in MakeDescriptor
    _message.default_pool.Add(file_descriptor_proto)
TypeError: Couldn't build proto file into descriptor pool!
Invalid proto descriptor for file "a8332bcbde22f967a60b51afdddf1cd5.proto":
  NanoPBOptions.long_names: "NanoPBOptions.long_names" is already defined in file "nanopb.proto".
  NanoPBOptions.packed_struct: "NanoPBOptions.packed_struct" is already defined in file "nanopb.proto".
  NanoPBOptions.packed_enum: "NanoPBOptions.packed_enum" is already defined in file "nanopb.proto".
  NanoPBOptions.skip_message: "NanoPBOptions.skip_message" is already defined in file "nanopb.proto".
  NanoPBOptions.no_unions: "NanoPBOptions.no_unions" is already defined in file "nanopb.proto".
  NanoPBOptions.anonymous_oneof: "NanoPBOptions.anonymous_oneof" is already defined in file "nanopb.proto".
  NanoPBOptions.proto3: "NanoPBOptions.proto3" is already defined in file "nanopb.proto".
  NanoPBOptions.enum_to_string: "NanoPBOptions.enum_to_string" is already defined in file "nanopb.proto".
  NanoPBOptions.fixed_length: "NanoPBOptions.fixed_length" is already defined in file "nanopb.proto".
  NanoPBOptions.fixed_count: "NanoPBOptions.fixed_count" is already defined in file "nanopb.proto".
  NanoPBOptions: "NanoPBOptions" is already defined in file "nanopb.proto".

解决方案：

pip uninstall protobuf
pip install --no-binary=protobuf protobuf

问题3: make失败2

Traceback (most recent call last):
  File "../../generator/nanopb_generator.py", line 50, in <module>
    from .proto import nanopb_pb2
ModuleNotFoundError: No module named '__main__.proto'; '__main__' is not a package

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "../../generator/nanopb_generator.py", line 71, in <module>
    import proto.nanopb_pb2 as nanopb_pb2
ModuleNotFoundError: No module named 'proto.nanopb_pb2'

请进入generator/proto目录下，执行***make all***命令

#nanopb-nanopb-0.3.9.2/generator/proto/Makefile

all: nanopb_pb2.py plugin_pb2.py

%_pb2.py: %.proto
        protoc --python_out=. $<

写在最后:
很明显可以发现编译.proto需要make 工具，请记得在编译前要保证必要的工具安装protocolbuffers

 sudo apt-get install autoconf automake libtool curl make g++ unzip

附nanopb-nanopb-0.3.9.2.zip, protobuf-all-3.13.0.tar.gz
参考来源
[1]: https://developers.google.com/protocol-buffers
[2]: https://github.com/protocolbuffers/protobuf
[3]: https://github.com/nanopb/nanopb