java flatbuffers_使用flatbuffers
问题
张三是个java程序员,他写产生数据的程序.李四是个python程序员,他要用python处理张三产生的数据.最直观常用的方法就是张三用java把产生的数据保存成csv或者xml文件,然后李四用python读取csv或xml文件.这没有问题.但现在有一种性能更好的方法,flatbuffers.
作用
可以把flatbuffers理解成一个可执行文件flatc.这个可执行文件可以把表示数据格式的fbs文件转化成c++,java,python...代码.张三擅长使用java,他就用flatc把fbs文件转化成java代码.用转化好的java代码把数据存成flatbuffers的格式.李四用flatc把同样的fbs文件转化成自己熟悉的python代码,李四可以用这份python代码把flatbuffers格式文件中的数据读出来.这就实现了不同语言间传数据.当然这只是一个简单的使用场景.flatbuffers的作用远不只是这些.
使用例子
先写一个表示数据格式的文件school.fbs
namespace school;
table School {
name:string;
students:[Student];
}
table Student {
number:short;
name:string;
}
root_type School;
具体的语法规则看参考资料,只简单的解释下.上面代码定义了一个根结点为School的数据.里面的student有2个成员变量number,name. School也有2个成员变量,一个是name代码school的名字.一个是Student的数组.
然后执行下面命令:
flatc.exe -p school.fbs
会生成一堆python代码.再用生成的python创建一个flatbuffers文件,代码如下：
# coding:utf-8
from __future__ import print_function
from __future__ import division
import flatbuffers
from school import School
from school import Student
builder = flatbuffers.Builder(1024)
stu1 = builder.CreateString('zhang3')
stu2 = builder.CreateString('li4')
Student.StudentStart(builder)
Student.StudentAddName(builder, stu1)
Student.StudentAddNumber(builder, 3)
zhang3 = Student.StudentEnd(builder)
Student.StudentStart(builder)
Student.StudentAddName(builder, stu2)
Student.StudentAddNumber(builder, 4)
li4 = Student.StudentEnd(builder)
School.SchoolStartStudentsVector(builder, 2)
builder.PrependUOffsetTRelative(li4)
builder.PrependUOffsetTRelative(zhang3)
students = builder.EndVector(2)
School.SchoolStart(builder)
School.SchoolAddStudents(builder, students)
school_result = School.SchoolEnd(builder)
builder.Finish(school_result)
buf = builder.Output()
new_school = School.School.GetRootAsSchool(buf, 0)
print(new_school.Students(0).Name())
print(new_school.Students(0).Number())
print(new_school.Students(1).Name())
print(new_school.Students(1).Number())
with open("result.bin", mode="wb") as f:
f.write(buf)
这段代码的作用是新建2个student，一个zhang3,一个li4,然后把student数据保存到School里,最后生成文件result.bin. result.bin就是flatbuffers格式的文件,本质下就是二进制文件.
再执行下面命令:
flatc.exe -j school.fbs
生成一堆java代码.可以用java代码把result.bin文件中的数据读出来.代码如下:
package com.company;
import school.School;
import java.io.*;
import java.nio.ByteBuffer;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
public class Main {
private void process() {
Path path = Paths.get("D:\\yourpath\\result.bin");
try {
byte[] data = Files.readAllBytes(path);
ByteBuffer buf = java.nio.ByteBuffer.wrap(data);
School school = School.getRootAsSchool(buf);
System.out.println(school.students(0).name());
System.out.println(school.students(0).number());
System.out.println(school.students(1).name());
System.out.println(school.students(1).number());
} catch (IOException e) {
System.out.println("IOException");
}
}
public static void main(String[] args) {
Main m = new Main();
m.process();
}
}
另一个名字
google在tensorflow lite中把result.bin这类二进制文件起一个高大上的名字——tflite
参考资料