TableStorage

Repository pattern with POCO object support for storing to Azure/CosmosDB Table Storage

Screenshot of basic usage

Usage

Given an entity like:

public record Product(string Category, string Id) 
{
  public string? Title { get; init; }
  public double Price { get; init; }
  public DateOnly CreatedAt { get; init; }
}

NOTE: entity can have custom constructor, key properties can be read-only, and it doesn’t need to inherit from anything, implement any interfaces or use any custom attributes (unless you want to). As shown above, it can even be a simple record type, with support for .NET 6 DateOnly type to boot!

The entity can be stored and retrieved with:

var storageAccount = CloudStorageAccount.DevelopmentStorageAccount; // or production one
// We lay out the parameter names for clarity only.
var repo = TableRepository.Create<Product>(storageAccount, 
    tableName: "Products",
    partitionKey: p => p.Category, 
    rowKey: p => p.Id);

var product = new Product("book", "1234") 
{
  Title = "Table Storage is Cool",
  Price = 25.5,
};

// Insert or Update behavior (aka "upsert")
await repo.PutAsync(product);

// Enumerate all products in category "book"
await foreach (var p in repo.EnumerateAsync("book"))
   Console.WriteLine(p.Price);

// Query books priced in the 20-50 range, 
// project just title + price
await foreach (var info in from prod in repo.CreateQuery()
                           where prod.Price >= 20 and prod.Price <= 50
                           select new { prod.Title, prod.Price })
  Console.WriteLine($"{info.Title}: {info.Price}");

// Get previously saved product.
Product saved = await repo.GetAsync("book", "1234");

// Delete product
await repo.DeleteAsync("book", "1234");

// Can also delete passing entity
await repo.DeleteAsync(saved);

If a unique identifier among all entities already exists, you can also store all entities in a single table, using a fixed partition key matching the entity type name, for example. In such a case, instead of a TableRepository, you can use a TablePartition:

public record Book(string ISBN, string Title, string Author, BookFormat Format, int Pages);

var storageAccount = CloudStorageAccount.DevelopmentStorageAccount; // or production one

// Leverage defaults: TableName=Entities, PartitionKey=Book
var repo = TablePartition.Create<Region>(storageAccount, 
  rowKey: book => book.ISBN);

// insert/get/delete same API shown above.

// query filtering by rowKey, in this case, books by a certain 
// language/publisher combination. For Disney/Hyperion in English, 
// for example: ISBNs starting with 978(prefix)-1(english)-4231(publisher)
var query = from book in repo.CreateQuery()
            where 
                book.ISBN.CompareTo("97814231") >= 0 &&
                book.ISBN.CompareTo("97814232") < 0
            select new { book.ISBN, book.Title };

await foreach (var book in query)
   ...

For the books example above, it might make sense to partition by author, for example. In that case, you could use a TableRepository<Book> when saving:

var repo = TableRepository.Create<Book>(storageAccount, "Books",
  partitionKey: x => x.Author, 
  rowKey: x => x.ISBN);

await repo.PutAsync(book);

And later on when listing/filtering books by a particular author, you can use a TablePartition<Book> so all querying is automatically scoped to that author:

var partition = TablePartition.Create<Book>(storageAccount, "Books", 
  partitionKey: "Rick Riordan", 
  rowKey: x => x.ISBN);

// Get Rick Riordan books, only from Disney/Hyperion, with over 1000 pages
var query = from book in repo.CreateQuery()
            where 
                book.ISBN.CompareTo("97814231") >= 0 &&
                book.ISBN.CompareTo("97814232") < 0 && 
                book.Pages >= 1000
            select new { book.ISBN, book.Title };

Using table partitions is quite convenient for handling reference data too, for example. Enumerating all entries in the partition wouldn’t be something you’d typically do for your “real” data, but for reference data, it could come in handy.

NOTE: if access to the Timestamp managed by Azure Table Storage for the entity is needed, just declare a property with that name with either DateTimeOffset, DateTime or string type to read it.

Stored entities using TableRepository and TablePartition use individual columns for properties, which makes it easy to browse the data (and query, as shown above!).

NOTE: partition and row keys can also be typed as Guid

But document-based storage is also available via DocumentRepository and DocumentPartition if you don’t need the individual columns.

Document Storage

The DocumentRepository.Create and DocumentPartition.Create factory methods provide access to document-based storage, exposing the a similar API as column-based storage.

Document repositories cause entities to be persisted as a single document column, alongside type and version information to handle versioning a the app level as needed.

The API is mostly the same as for column-based repositories (document repositories implement the same underlying ITableStorage interface):

public record Product(string Category, string Id) 
{
  public string? Title { get; init; }
  public double Price { get; init; }
  public DateOnly CreatedAt { get; init; }
}

var book = new Product("book", "9781473217386")
{
    Title = "Neuromancer",
    Price = 7.32
};

// Column-based storage
var repo = TableRepository.Create<Product>(
    CloudStorageAccount.DevelopmentStorageAccount,
    tableName: "Products",
    partitionKey: p => p.Category,
    rowKey: p => p.Id);

await repo.PutAsync(book);

// Document-based storage
var docs = DocumentRepository.Create<Product>(
    CloudStorageAccount.DevelopmentStorageAccount,
    tableName: "Documents",
    partitionKey: p => p.Category,
    rowKey: p => p.Id
    serializer: [SERIALIZER]);

await docs.PutAsync(book);

If not provided, the serializer defaults to the System.Text.Json-based DocumentSerializer.Default.

The resulting differences in storage can be seen in the following screenshots of the Azure Storage Explorer:

Screenshot of entity persisted with separate columns for properties

Screenshot of entity persisted as a document

The Type column persisted in the documents table is the Type.FullName of the persisted entity, and the Version is the [Major].[Minor] of its assembly, which could be used for advanced data migration scenarios. The major and minor version components are also provided as individual columns for easier querying by various version ranges, using IDocumentRepository.EnumerateAsync(predicate).

In addition to the default built-in JSON plain-text based serializer (which uses the System.Text.Json package), there are other alternative serializers for the document-based repository, including various binary serializers which will instead persist the document as a byte array:

You can pass the serializer to use to the factory method as follows:

var repo = TableRepository.Create<Product>(...,
    serializer: [JsonDocumentSerializer|BsonDocumentSerializer|MessagePackDocumentSerializer|ProtobufDocumentSerializer].Default);

NOTE: when using alternative serializers, entities might need to be annotated with whatever attributes are required by the underlying libraries.

Attributes

If you want to avoid using strings with the factory methods, you can also annotate the entity type to modify the default values used:

[Table("tableName")]: class-level attribute to change the default when no value is provided
[PartitionKey]: annotates the property that should be used as the partition key
[RowKey]: annotates the property that should be used as the row key.

Values passed to the factory methods override declarative attributes.

For the products example above, your record entity could be:

[Table("Products")]
public record Product([PartitionKey] string Category, [RowKey] string Id) 
{
  public string? Title { get; init; }
  public double Price { get; init; }
}

And creating the repository wouldn’t require any arguments besides the storage account:

var repo = TableRepository.Create<Product>(CloudStorageAccount.DevelopmentStorageAccount);

In addition, if you want to omit a particular property from persistence, you can annotate it with [Browsable(false)] and it will be skipped when persisting and reading the entity.

TableEntity Support

Since these repository APIs are quite a bit more intuitive than working directly against a
TableClient, you might want to retrieve/enumerate entities just by their built-in TableEntity properties, like PartitionKey, RowKey, Timestamp and ETag. For this scenario, we also support creating ITableRepository<TableEntity> and ITablePartition<TableEntity> by using the factory methods TableRepository.Create(...) and TablePartition.Create(...) without a (generic) entity type argument.

For example, given you know all Region entities saved in the example above, use the region Code as the RowKey, you could simply enumerate all regions without using the Region type at all:

var account = CloudStorageAccount.DevelopmentStorageAccount; // or production one
var repo = TablePartition.Create(storageAccount, 
  tableName: "Reference",
  partitionKey: "Region");

// Enumerate all regions within the partition as plain TableEntities
await foreach (TableEntity region in repo.EnumerateAsync())
   Console.WriteLine(region.RowKey);

You can access and add additional properties by just using the entity indexer, which you can later persist by calling PutAsync:

await repo.PutAsync(
    new TableEntity("Book", "9781473217386") 
    {
        ["Title"] = "Neuromancer",
        ["Price"] = 7.32
    });

var entity = await repo.GetAsync("Book", "9781473217386");

Assert.Equal("Neuromancer", entity["Title"]);
Assert.Equal(7.32, (double)entity["Price"]);

Installation

> Install-Package Devlooped.TableStorage

All packages also come in source-only versions, if you want to avoid an additional assembly dependency:

> Install-Package Devlooped.TableStorage.Source

The source-only packages includes all types with the default visibility (internal), so you can decide what types to make public by declaring a partial class with the desired visibility. To make them all public, for example, you can include the same Visibility.cs that the compiled version uses.