• en | es

Testing Spark Streaming: Integration testing with Docker Compose

In the first post of this series, we saw how to unit test Spark Streaming operations using Spark Testing Base. Here we'll see how to do integration testing using Docker Compose. What is Integration testing We previously saw a discussion about unit and integration testing....


Testing Spark Streaming: Unit testing

There is enough evidence to prove the importance of automated testing. Projects in new fields often neglect automated testing, as the domain itself steals the attention of the developers. However, lack of testing implies 'laugh now, cry later'. Some of the tools around Big Data...