File storage is an important feature required in multiple processes across various types of applications. The existence of processes like Content Delivery Networks (CDNs)
, set up through third-party cloud options like Amazon Web Services, and local file storage options have always made it easier to build such a feature.
文件存储是跨各种类型的应用程序的多个进程所需的一项重要功能。 诸如Content Delivery Networks (CDNs)
类的过程的存在,通过诸如Amazon Web Services之类的第三方云选项设置的过程以及本地文件存储选项的存在,始终使构建此类功能变得更加容易。
However, the concept of storing files directly into a database through a single API call had intrigued me for quite some time. That is where GridFS came into the picture for me.
但是,通过单个API调用将文件直接存储到数据库中的概念让我很感兴趣。 那就是GridFS成为我关注的地方。
MongoDB has a driver specification to upload and retrieve files from it called GridFS. GridFS allows you to store and retrieve files, which includes ones exceeding the BSON-document size limit of 16 MB.
MongoDB有一个驱动程序规范,称为GridFS,用于从中上传和检索文件。 GridFS允许您存储和检索文件,其中包括超出BSON文档大小限制16 MB的文件 。
GridFS basically takes a file and breaks it up into multiple chunks which are stored as individual documents in two collections:
GridFS基本上获取一个文件并将其分成多个块,这些块作为单独的文档存储在两个集合中:
the chunk
collection (stores the document parts), and
chunk
集合 (存储文档部分),以及
the file
collection (stores the consequent additional metadata).
file
集合 (存储随后的其他元数据)。
Each chunk is limited to 255 KB in size. This means that the last chunk is normally either equal to or less than 255 KB. Sounds rather neat.
每个块的大小限制为255 KB。 这意味着最后一个块通常等于或小于255 KB。 听起来很整洁。
When you read from GridFS, the driver reassembles all the chunks as needed. This means that you can read sections of a file as per your query range. Such as listening to a segment of an audio file or fetching a section of a video file.
当您从GridFS中读取时,驱动程序将根据需要重新组装所有块。 这意味着您可以根据查询范围读取文件的各个部分。 例如,听一段音频文件或获取一段视频文件。
Note: It is preferred to use GridFS for storing files normally exceeding the 16 MB size limit. For smaller files, it is recommended to use the BinData format to store the files in single documents.
注意:最好使用GridFS来存储通常超过16 MB大小限制的文件。 对于较小的文件,建议使用BinData格式将文件存储在单个文档中。
This summarizes how GridFS works in general. Time to dip our feet into some working code and see how to implement a system as such.
这概括了GridFS的总体工作方式。 是时候花些时间编写一些有效的代码,看看如何实现这样的系统了。
We are using Node.js with access to a cloud instance of MongoDB for our setup. You can find the code repository for the sample application here.
我们使用Node.js来访问MongoDB的云实例以进行设置。 您可以在此处找到示例应用程序的代码存储库。
We will completely focus on segments of the code that relate to the functionalities of GridFS. We'll learn how to set it up and use it to store files, retrieve files or a particular file, and delete a particular file. Let's start then.
我们将完全专注于与GridFS功能相关的代码段。 我们将学习如何设置它并使用它来存储文件,检索文件或特定文件以及删除特定文件。 让我们开始吧。
The packages needed to initialize the engine are multer-gridfs-storage
and multer
. We also use method-override
middleware to enable the delete operation for files. The npm module crypto
is used to encrypt the filenames on being stored and read from the database.
初始化引擎所需的软件包是multer-gridfs-storage
和multer
。 我们还使用method-override
中间件来启用文件的删除操作。 npm模块crypto
用于在存储和从数据库读取文件名时对其进行加密。
Once the storage engine using GridFS is initialized, you have to just call it using the multer middleware. It is then passed to the respective route executing the various file storage operations.
初始化使用GridFS的存储引擎后,您只需使用multer中间件进行调用。 然后将其传递到执行各种文件存储操作的相应路由。
We initialize a GridFS stream as seen in the code below. The stream is needed to read the files from the database and also to help render an image to a browser when needed.
如下面的代码所示,我们初始化了GridFS流。 需要该流以从数据库读取文件,并在需要时帮助将图像呈现给浏览器。
We reuse the upload middleware we had created earlier.
我们重用了之前创建的上传中间件。
Note: The name file
is used as a parameter in upload.single()
since we have the key with a similar name carrying the file being sent from the client.
注意:名称file
在upload.single()
用作参数,因为我们有一个名称相似的密钥,该密钥包含从客户端发送的文件。
We can also upload multiple files at once. Instead of upload.single()
, we have to simply use upload.multiple(<number of files>)
.
我们也可以一次上传多个文件。 代替upload.single()
,我们必须简单地使用upload.multiple(<number of files>)
。
Note: The number of files uploaded can be less than the defined number of files.
注意:上载的文件数可以少于定义的文件数。
Using the stream initialized we can fetch all the files in the particular database using gfs.find().toArray(...)
. Once the files are obtained we map it to an array and ship the response.
使用初始化的流,我们可以使用gfs.find().toArray(...)
获取特定数据库中的所有文件。 获取文件后,我们将其映射到数组并发送响应。
It is super simple to query GridFS for a single file based on a specific attribute like filename
. Using the GridFS stream, you can query the database through the function gfs.find({<add query here>})
.
根据特定属性(如filename
为单个文件查询GridFS超级简单。 使用GridFS流,您可以通过函数gfs.find({<add query here>})
查询数据库。
This is a slightly trickier part since you have to not only fetch a file from the database but also to render it as an image on the respective browser. We fetch the file normally. No change in that process.
这是一个比较棘手的部分,因为您不仅必须从数据库中获取文件,而且还必须在相应的浏览器中将其呈现为图像。 我们正常获取文件。 在此过程中没有任何变化。
Then with the help of the method openDownloadStreamByName()
on gfs stream, we can easily render an image as it returns a readable stream. Having done that, we can use JavaScript's pipe()
to stream the response.
然后借助gfs流上的openDownloadStreamByName()
方法,当图像返回可读流时,我们可以轻松地渲染图像。 完成此操作后,我们可以使用JavaScript的pipe()
流响应。
Deleting a file is equally straight-forward. We use the stream method delete()
with _id
parameter to query and delete the concerned file.
同样,删除文件也很简单。 我们使用带有_id
参数的流方法delete()
来查询和删除相关文件。
Those are the major functionalities offered by the storage engine design. I had leveraged the GridFS features discussed to create a simple image uploading application. You can delve deeper into the code in the respository.
这些是存储引擎设计提供的主要功能。 我利用所讨论的GridFS功能创建了一个简单的图像上传应用程序。 你可以深入探究的代码程序存储库 。
It took me some time and a decent amount of struggle to understand how to make use of GridFS for a personal project. Because of this, I wanted to make sure that at least one other person didn't have to invest the same amount of time.
我花了一些时间和大量的精力来理解如何将GridFS用于个人项目。 因此,我想确保至少另一个人不必花费相同的时间。
Having said that, I would recommend using GridFS with caution. It is not a silver bullet to all your file storage concerns. Still, it is a nifty specification to know and be aware of.
话虽如此,我建议谨慎使用GridFS。 这并不是解决所有文件存储问题的灵丹妙药 。 尽管如此,这仍然是一个不错的规范。
If you have any queries or concerns, you can comment in the post or reach out to me on LinkedIn
.
如果您有任何疑问或疑虑,可以在帖子中发表评论,或在LinkedIn
上与我联系。
In the mean time, keep coding.
同时,继续编码。
翻译自: https://www.freecodecamp.org/news/gridfs-making-file-uploading-to-mongodb/