VMware vFabric GemFire 6.5: modern data management for modern applications

Engineering
Adam Fitzgerald
October 12, 2010

Just some of the properties of new, modern applications are that they tend to be highly interactive, data rich, include business processes, and are available over the web from PCs and mobile devices. Powering these applications are platforms that leverage virtualized infrastructure, provide just the right amount of functionality and can scale easily as demand shifts.

VMware’s vFabric Cloud Application Platform is just such a platform designed to run Spring applications most efficiently and deliver the modern application experience. An essential component of this platform is vFabric GemFire. vFabric GemFire in-memory distributed data management brings data to applications with low latency, high reliability and linear, dynamic scalability. We are pleased to announce vFabric GemFire version 6.5.

vFabric GemFire chief architect, Jags Ramnarayan, provides all of the details about the release in his latest blog post, but some of key new capabilities include:

Simplified and Intuitive programming model: First, we simplified the programming model by reducing some of the boiler plate bootstrapping code that was required in the past and introduced pre-packaged templates for common configurations and topologies. Second, we launched the new Spring-Gemfire project to introduce a clean programming model for the Spring developer. Note that Spring-GemFire is not bundled with GemFire 6.5. If you are already familiar with the GemFire APIs or just getting started, I would recommend going through the new tutorial that walks through some of the new simplified programming APIs. And, not to worry - your existing application will continue to just run fine. The old APIs are fully supported.
Database class reliability through Shared-nothing parallel persistence: A unique high performance design to pool disks across the cluster for storing and recovering data. GemFire always supported disk persistence and recovery for replicated data regions but now this capability has been extended for partitioned data also. The design principles adopted are fundamentally different than ones in typical clustered databases. For one, disk storage is shared nothing - each cache process owns its disk store eliminating process level contentions. Second, the design is tilted to favor memory i.e. there are no complex B-Tree data structures on disk; instead we assume complex query navigations will always be done through in-memory indexes. Third, the design uses rolling append-only log files to avoid disk seeks completely. Finally, the design preserves the rebalancing model in GemFire when capacity is increased or decreased - the disk data also relocates itself.
Improved scale-out capabilities: Application deployments using the tiered model (client process embedding a local cache talking to a server farm) could see a 2X or more performance gain when accessing partitioned data. With 6.5, client processes gain knowledge about server side partitions and use it on the fly to direct traffic directly to the server with the required data set. Application clients subscribing to events using key based register interest or "continuous queries" now have their subscriptions registered on all the partitioned nodes. This allows each partition to process the subcription filters on the nodes where the data changes are applied dramatically reducing messaging traffic between peers unlike in the previous releases. The net effect is that more subscriptions can be processed and the event delivery latency to clients is also reduced.
Colocated transactions: If each logical partition were to own the entire transactional working set then highly applications can scale linearly if the concurrent transaction load is uniformly spread across the data set and hence across all the partitions. Each partition can coordinate its transaction without any impact to other partitions with no locking requirements across partitions. GemFire 6.5 introduces a change to the distributed transaction design to detect cases where the data is colocated and avoids engaging the built-in distributed lock service.

The complete list of features added in 6.5, can be found in the Gemfire documentation. Also the blog post by Jags covers a lot more detail about th factors contributing to very high disk throughput. Download the new Gemfire 6.5 and take it for a spin.
comments powered by Disqus

translate：
翻译：

新的现代应用程序的一些特性是，它们往往具有高度的交互性、丰富的数据、包括业务流程，并且可以从个人电脑和移动设备上通过web访问。为这些应用程序提供动力的平台利用了虚拟化基础设施，提供了恰到好处的功能，并且可以随着需求的变化轻松扩展。
VMware的vFabric云应用程序平台正是这样一个平台，旨在最高效地运行Spring应用程序并提供现代应用程序体验。这个平台的一个重要组成部分是vFabric GemFire。vFabric GemFire内存分布式数据管理以低延迟、高可靠性和线性、动态可扩展性为应用带来数据。我们很高兴宣布vFabric GemFire版本6.5条.
vFabric GemFire首席架构师Jags Ramnarayan在其最新的博客文章中提供了关于发布的所有细节，但一些关键的新功能包括：
简化和直观的编程模型：首先，我们简化了编程模型，减少了过去需要的一些锅炉板引导代码，并为常见配置和拓扑引入了预打包模板。其次，我们启动了新的Spring Gemfire项目，为Spring开发人员引入了一个干净的编程模型。请注意，弹簧GemFire未与GemFire捆绑在一起6.5条. 如果您已经熟悉gemfireapi或者刚刚开始，我建议您阅读新教程，其中介绍了一些新的简化编程api。而且，不用担心-您的现有应用程序将继续正常运行。旧的api完全受支持。
通过不共享任何内容的并行持久性实现的数据库类可靠性：一种独特的高性能设计，用于在集群中共享磁盘以存储和恢复数据。GemFire始终支持复制数据区域的磁盘持久性和恢复，但现在这种功能也扩展到了分区数据。所采用的设计原则与典型的集群数据库中的设计原则有根本的不同。首先，磁盘存储不共享任何内容—每个缓存进程都拥有其磁盘存储，从而消除了进程级的争用。第二，设计倾向于记忆即。磁盘上没有复杂的B树数据结构；相反，我们假设复杂的查询导航将始终通过内存索引完成。第三，设计采用滚动只追加日志文件的方式，完全避免了磁盘查找。最后，当容量增加或减少时，该设计保留了GemFire中的重新平衡模型-磁盘数据也会自行重新定位。
改进的扩展功能：使用分层模型（客户端进程嵌入本地缓存并与服务器场对话）的应用程序部署在访问分区数据时可以获得2倍或更多的性能提升。与6.5条，客户机进程获得有关服务器端分区的知识，并动态使用它将通信量直接引导到具有所需数据集的服务器。使用基于键的注册兴趣或“连续查询”订阅事件的应用程序客户端现在已在所有分区节点上注册其订阅。这允许每个分区在应用数据更改的节点上处理子脚本过滤器，与以前的版本不同，这大大减少了对等端之间的消息通信量。最终的效果是，可以处理更多的订阅，同时也减少了对客户端的事件传递延迟。
共中心事务：如果每个逻辑分区拥有整个事务工作集，那么如果并发事务负载均匀地分布在数据集上，从而分布在所有分区上，那么高度应用程序可以线性地扩展。每个分区都可以协调其事务，而不会对其他分区造成任何影响，并且分区之间没有锁定要求。金火6.5条引入对分布式事务设计的更改，以检测数据同位的情况，并避免使用内置的分布式锁服务。