Introduction

This time I would like to describe how we can protect our REST API applications from requests containing invalid data (data validation process). However, validation of our requests is not enough, unfortunately. In addition to validation, it is our responsibility to return the relevant messages and statuses to our API clients. I wanted to deal with these two things in this post.

Data Validation

Definition of Data Validation

What is data validation really? The best definition I found is from UNECE Data Editing Group:

An activity aimed at verifying whether the value of a data item comes from the given (finite or infinite) set of acceptable values.

According to this definition we should verify data items which are coming to our application from external sources and check if theirs values are acceptable. How do we know that the value is acceptable? We need to define data validation rules for every type of data item which is processing in our system.

Data vs Business Rules validation

I would like to emphasize that data validation is totally different concept than validation of business rules. Data validation is focused on verifying an atomic data item. Business rules validation is a more broad concept and more close to how business works and behaves. So it is mainly focused on behavior. Of course validating behavior depends on data too, but in a more wide range.

Examples of data validation:

– Product order quantity cannot be negative or zero

– Product order quantity should be a number

– Currency of order should be a value from currencies list

Examples of business rules validation

– Product can be ordered only when Customer age is equal or greater than product minimal age.

– Customer can place only two orders in one day.

Returning relevant information

If we acknowledge that the rules have been broken during validation, we must stop processing and return the equivalent message to the client. We should follow the following rules:

– we should return message to the client as fast as possible (Fail-fast principle)

– the reason for the validation error should be well explained and understood for the client

– we should not return technical aspects for security reasons

Problem Details for HTTP APIs standard

The issue of returned error messages is so common that a special standard was created describing how to handle such situations. It is called “Problem Details for HTTP APIs standard” and his official description can be found here. This is abstract of this standard:

This document defines a “problem detail” as a way to carry machine-readable details of errors in a HTTP response to avoid the need to define new error response formats for HTTP APIs.

Problem Details standard introduces Problem Details JSON object, which should be part of the response when validation error occurs. This is simple canonical model with 5 members:

– problem type

– title

– HTTP status code

– details of error

– instance (pointer to specific occurrence)

Of course we can (and sometimes we should) extend this object by adding new properties, but the base should be the same. Thanks to this our API is easier to understand, learn and use. For more detailed information about standard I invite you to read documentation which is well described.

Data validation localization

For the standard application we can put data validation logic in three places:

GUI – it is entry point for users input. Data is validated on the client side, for example using Javascript for web applications

Application logic/services layer – data is validated in specific application service or command handler on the server side

Database – this is exit point of request processing and last moment to validate the data

In this article I am omitting GUI and Database components and I am focusing on the server side of the application. Let’s see how we can implement data validation on Application Services layer.

Implementing Data Validation

Suppose we have a command AddCustomerOrderCommand:

AddCustomerOrderCommand Command public class AddCustomerOrderCommand : IRequest { public Guid CustomerId { get; } public List<ProductDto> Products { get; } public AddCustomerOrderCommand( Guid customerId, List<ProductDto> products) { this.CustomerId = customerId; this.Products = products; } } public class ProductDto { public Guid Id { get; set; } public int Quantity { get; set; } public string Currency { get; set; } public string Name { get; set; } } 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 public class AddCustomerOrderCommand : IRequest { public Guid CustomerId { get ; } public List < ProductDto > Products { get ; } public AddCustomerOrderCommand ( Guid customerId , List < ProductDto > products ) { this . CustomerId = customerId ; this . Products = products ; } } public class ProductDto { public Guid Id { get ; set ; } public int Quantity { get ; set ; } public string Currency { get ; set ; } public string Name { get ; set ; } }

Suppose we want to validate 4 things:

1. CustomerId is not empty GUID.

2. Products list is not empty

3. Each product quantity is greater than 0

4. Each product currency is equal to USD or EUR

Let me show 3 solutions to this problem – from simple to the most sophisticated.

1. Simple validation on Application Service

The first thing that can come to mind is a simple validation in the Command Handler itself. In this solution we need to implement private method which validates our command and throws exception if validation error occurs. Closing this kind of logic in separate method is better from the Clean Code perspective (see Extract Method too).

Validate method public class AddCustomerOrderCommandHandler : IRequestHandler<AddCustomerOrderCommand> { private readonly ICustomerRepository _customerRepository; private readonly IProductRepository _productRepository; public AddCustomerOrderCommandHandler( ICustomerRepository customerRepository, IProductRepository productRepository) { this._customerRepository = customerRepository; this._productRepository = productRepository; } public async Task<Unit> Handle(AddCustomerOrderCommand request, CancellationToken cancellationToken) { Validate(request); var customer = await this._customerRepository.GetByIdAsync(request.CustomerId); // logic.. } private static void Validate(AddCustomerOrderCommand command) { var errors = new List<string>(); if (command.CustomerId == Guid.Empty) { errors.Add("CustomerId is empty"); } if (command.Products == null || !command.Products.Any()) { errors.Add("Products list is empty"); } else { if (command.Products.Any(x => x.Quantity < 1)) { errors.Add("At least one product has invalid quantity"); } if (command.Products.Any(x => x.Currency != "USD" && x.Currency != "EUR")) { errors.Add("At least one product has invalid currency"); } } if (errors.Any()) { var errorBuilder = new StringBuilder(); errorBuilder.AppendLine("Invalid order, reason: "); foreach (var error in errors) { errorBuilder.AppendLine(error); } throw new Exception(errorBuilder.ToString()); } } } 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 public class AddCustomerOrderCommandHandler : IRequestHandler < AddCustomerOrderCommand > { private readonly ICustomerRepository _customerRepository ; private readonly IProductRepository _productRepository ; public AddCustomerOrderCommandHandler ( ICustomerRepository customerRepository , IProductRepository productRepository ) { this . _customerRepository = customerRepository ; this . _productRepository = productRepository ; } public async Task < Unit > Handle ( AddCustomerOrderCommand request , CancellationToken cancellationToken ) { Validate ( request ) ; var customer = await this . _customerRepository . GetByIdAsync ( request . CustomerId ) ; // logic.. } private static void Validate ( AddCustomerOrderCommand command ) { var errors = new List < string > ( ) ; if ( command . CustomerId == Guid . Empty ) { errors . Add ( "CustomerId is empty" ) ; } if ( command . Products == null || ! command . Products . Any ( ) ) { errors . Add ( "Products list is empty" ) ; } else { if ( command . Products . Any ( x = > x . Quantity < 1 ) ) { errors . Add ( "At least one product has invalid quantity" ) ; } if ( command . Products . Any ( x = > x . Currency != "USD" && x . Currency != "EUR" ) ) { errors . Add ( "At least one product has invalid currency" ) ; } } if ( errors . Any ( ) ) { var errorBuilder = new StringBuilder ( ) ; errorBuilder . AppendLine ( "Invalid order, reason: " ) ; foreach ( var error in errors ) { errorBuilder . AppendLine ( error ) ; } throw new Exception ( errorBuilder . ToString ( ) ) ; } } }

The result of invalid command execution:



This is not so bad approach but has two disadvantages. Firstly, it involves from us writing a lot of easy and boilerplate code – comparing to nulls, defaults, values from list etc. Secondly, we are losing here part of separation of concerns because we are mixing validation logic with orchestrating our use case flow. Let’s take care of boilerplate code first.

2. Validation using FluentValidation library

We don’t want to reinvent the wheel so the best solution is to use library. Fortunately, there is a great library for validation in .NET world – Fluent Validation. It has nice API and a lot of features. This is how we can use it to validate our command:

FluentValidation validator public class AddCustomerOrderCommandValidator : AbstractValidator<AddCustomerOrderCommand> { public AddCustomerOrderCommandValidator() { RuleFor(x => x.CustomerId).NotEmpty().WithMessage("CustomerId is empty"); RuleFor(x => x.Products).NotEmpty().WithMessage("Products list is empty"); RuleForEach(x => x.Products).SetValidator(new ProductDtoValidator()); } } public class ProductDtoValidator : AbstractValidator<ProductDto> { public ProductDtoValidator() { this.RuleFor(x => x.Currency).Must(x => x == "USD" || x == "EUR") .WithMessage("At least one product has invalid currency"); this.RuleFor(x => x.Quantity).GreaterThan(0) .WithMessage("At least one product has invalid quantity"); } } 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 public class AddCustomerOrderCommandValidator : AbstractValidator < AddCustomerOrderCommand > { public AddCustomerOrderCommandValidator ( ) { RuleFor ( x = > x . CustomerId ) . NotEmpty ( ) . WithMessage ( "CustomerId is empty" ) ; RuleFor ( x = > x . Products ) . NotEmpty ( ) . WithMessage ( "Products list is empty" ) ; RuleForEach ( x = > x . Products ) . SetValidator ( new ProductDtoValidator ( ) ) ; } } public class ProductDtoValidator : AbstractValidator < ProductDto > { public ProductDtoValidator ( ) { this . RuleFor ( x = > x . Currency ) . Must ( x = > x == "USD" || x == "EUR" ) . WithMessage ( "At least one product has invalid currency" ) ; this . RuleFor ( x = > x . Quantity ) . GreaterThan ( 0 ) . WithMessage ( "At least one product has invalid quantity" ) ; } }

Now, the Validate method looks like:

Validate with FluentValidation private static void Validate(AddCustomerOrderCommand command) { AddCustomerOrderCommandValidator validator = new AddCustomerOrderCommandValidator(); var validationResult = validator.Validate(command); if (!validationResult.IsValid) { var errorBuilder = new StringBuilder(); errorBuilder.AppendLine("Invalid order, reason: "); foreach (var error in validationResult.Errors) { errorBuilder.AppendLine(error.ErrorMessage); } throw new Exception(errorBuilder.ToString()); } } 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 private static void Validate ( AddCustomerOrderCommand command ) { AddCustomerOrderCommandValidator validator = new AddCustomerOrderCommandValidator ( ) ; var validationResult = validator . Validate ( command ) ; if ( ! validationResult . IsValid ) { var errorBuilder = new StringBuilder ( ) ; errorBuilder . AppendLine ( "Invalid order, reason: " ) ; foreach ( var error in validationResult . Errors ) { errorBuilder . AppendLine ( error . ErrorMessage ) ; } throw new Exception ( errorBuilder . ToString ( ) ) ; } }

The result of validation is the same as earlier, but now our validation logic is more cleaner. The last thing to do is decouple this logic from Command Handler completely…

3. Validation using Pipeline Pattern

To decouple validation logic and execute it before Command Handler execution we arrange our command handling process in Pipeline (see NServiceBus Pipeline also).

For the Pipeline implementation we can use easily MediatR Behaviors. First thing to do is behavior implementation:

CommandValidationBehavior public class CommandValidationBehavior<TRequest, TResponse> : IPipelineBehavior<TRequest, TResponse> { private readonly IList<IValidator<TRequest>> _validators; public CommandValidationBehavior(IList<IValidator<TRequest>> validators) { this._validators = validators; } public Task<TResponse> Handle(TRequest request, CancellationToken cancellationToken, RequestHandlerDelegate<TResponse> next) { var errors = _validators .Select(v => v.Validate(request)) .SelectMany(result => result.Errors) .Where(error => error != null) .ToList(); if (errors.Any()) { var errorBuilder = new StringBuilder(); errorBuilder.AppendLine("Invalid command, reason: "); foreach (var error in errors) { errorBuilder.AppendLine(error.ErrorMessage); } throw new Exception(errorBuilder.ToString()); } return next(); } } 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 public class CommandValidationBehavior < TRequest , TResponse > : IPipelineBehavior < TRequest , TResponse > { private readonly IList < IValidator < TRequest >> _validators ; public CommandValidationBehavior ( IList < IValidator < TRequest >> validators ) { this . _validators = validators ; } public Task < TResponse > Handle ( TRequest request , CancellationToken cancellationToken , RequestHandlerDelegate < TResponse > next ) { var errors = _validators . Select ( v = > v . Validate ( request ) ) . SelectMany ( result = > result . Errors ) . Where ( error = > error != null ) . ToList ( ) ; if ( errors . Any ( ) ) { var errorBuilder = new StringBuilder ( ) ; errorBuilder . AppendLine ( "Invalid command, reason: " ) ; foreach ( var error in errors ) { errorBuilder . AppendLine ( error . ErrorMessage ) ; } throw new Exception ( errorBuilder . ToString ( ) ) ; } return next ( ) ; } }

Next thing to do is to register behavior in IoC container (Autofac example):

Register CommandValidationBehavior public class MediatorModule : Autofac.Module { protected override void Load(ContainerBuilder builder) { builder.RegisterAssemblyTypes(typeof(IMediator).GetTypeInfo().Assembly).AsImplementedInterfaces(); var mediatrOpenTypes = new[] { typeof(IRequestHandler<,>), typeof(INotificationHandler<>), typeof(IValidator<>), }; foreach (var mediatrOpenType in mediatrOpenTypes) { builder .RegisterAssemblyTypes(typeof(GetCustomerOrdersQuery).GetTypeInfo().Assembly) .AsClosedTypesOf(mediatrOpenType) .AsImplementedInterfaces(); } builder.RegisterGeneric(typeof(RequestPostProcessorBehavior<,>)).As(typeof(IPipelineBehavior<,>)); builder.RegisterGeneric(typeof(RequestPreProcessorBehavior<,>)).As(typeof(IPipelineBehavior<,>)); builder.Register<ServiceFactory>(ctx => { var c = ctx.Resolve<IComponentContext>(); return t => c.Resolve(t); }); builder.RegisterGeneric(typeof(CommandValidationBehavior<,>)).As(typeof(IPipelineBehavior<,>)); } } 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 public class MediatorModule : Autofac . Module { protected override void Load ( ContainerBuilder builder ) { builder . RegisterAssemblyTypes ( typeof ( IMediator ) . GetTypeInfo ( ) . Assembly ) . AsImplementedInterfaces ( ) ; var mediatrOpenTypes = new [ ] { typeof ( IRequestHandler < , > ) , typeof ( INotificationHandler <> ) , typeof ( IValidator <> ) , } ; foreach ( var mediatrOpenType in mediatrOpenTypes ) { builder . RegisterAssemblyTypes ( typeof ( GetCustomerOrdersQuery ) . GetTypeInfo ( ) . Assembly ) . AsClosedTypesOf ( mediatrOpenType ) . AsImplementedInterfaces ( ) ; } builder . RegisterGeneric ( typeof ( RequestPostProcessorBehavior < , > ) ) . As ( typeof ( IPipelineBehavior < , > ) ) ; builder . RegisterGeneric ( typeof ( RequestPreProcessorBehavior < , > ) ) . As ( typeof ( IPipelineBehavior < , > ) ) ; builder . Register < ServiceFactory > ( ctx = > { var c = ctx . Resolve < IComponentContext > ( ) ; return t = > c . Resolve ( t ) ; } ) ; builder . RegisterGeneric ( typeof ( CommandValidationBehavior < , > ) ) . As ( typeof ( IPipelineBehavior < , > ) ) ; } }

This way we achieved separation of concerns and Fail-fast principle implementation in nice and elegant way.

But this is not the end. Finally, we need to do something with returned messages to clients.

Implementing Problem Details standard

Just as in the case of validation logic implementation, we will use a dedicated library – ProblemDetails. The principle of the mechanism is simple. Firstly, we need to create custom exception:

InvalidCommandException public class InvalidCommandException : Exception { public string Details { get; } public InvalidCommandException(string message, string details) : base(message) { this.Details = details; } } 1 2 3 4 5 6 7 8 public class InvalidCommandException : Exception { public string Details { get ; } public InvalidCommandException ( string message , string details ) : base ( message ) { this . Details = details ; } }

Secondly, we have to create own Problem Details class:

InvalidCommandProblemDetails public class InvalidCommandProblemDetails : Microsoft.AspNetCore.Mvc.ProblemDetails { public InvalidCommandProblemDetails(InvalidCommandException exception) { this.Title = exception.Message; this.Status = StatusCodes.Status400BadRequest; this.Detail = exception.Details; this.Type = "https://somedomain/validation-error"; } } 1 2 3 4 5 6 7 8 9 10 public class InvalidCommandProblemDetails : Microsoft . AspNetCore . Mvc . ProblemDetails { public InvalidCommandProblemDetails ( InvalidCommandException exception ) { this . Title = exception . Message ; this . Status = StatusCodes . Status400BadRequest ; this . Detail = exception . Details ; this . Type = "https://somedomain/validation-error" ; } }

Last thing to do is to add Problem Details Middleware with definition of mapping between InvalidCommandException and InvalidCommandProblemDetails class in startup:

Startup services.AddProblemDetails(x => { x.Map<InvalidCommandException>(ex => new InvalidCommandProblemDetails(ex)); }); .... app.UseProblemDetails(); 1 2 3 4 5 6 7 8 services . AddProblemDetails ( x = > { x . Map < InvalidCommandException > ( ex = > new InvalidCommandProblemDetails ( ex ) ) ; } ) ; . . . . app . UseProblemDetails ( ) ;

After change in CommandValidationBehavior (throwing InvalidCommandExecption instead Exception) we have returned content compatible with the standard:

Summary

In this post I described:

– what Data validation is and where is located

– what Problem Details for HTTP APIs is and how could be implemented

– 3 methods to implement data validation in Application Services layer: without any patterns and tools, with FluentValidation library, and lastly – using Pipeline Pattern and MediatR Behaviors.

Source code

If you would like to see full, working example – check my GitHub repository

Related posts

Domain Model Encapsulation and PI with Entity Framework 2.2

Simple CQRS implementation with raw SQL and DDD

How to publish and handle Domain Events

10 common broken rules of clean code