LOGIN
התחברות או הרשמה
Avatar
להמשך הרשמה ידנית – לחץ על כפתור ההרשמה, להרשמה/כניסה מהירה בעזרת חשבון רשת חברתית – לחץ על הלוגו בכותרת

אפס סיסמה - שכחתי את שם המשתמש

שם משתמש
סיסמה
זכור אותי

he icon   en icon

עליכם להיכנס כמשתמשים רשומים בכדי להשתתף בפעילויות הקהילה - ההרשמה/כניסה מתבצעת מכותרת האתר.

הקהילה

Gidon Benuri תמונות - index.

  • 0
בתמונה זו:

חדשות מעולם הבדיקות

  • Streaming Kafka topic to Delta table (S3) with Spark Structured Streaming

    Streaming a Kafka topic in a Delta table on S3 using Spark Structured StreamingAt Wehkamp we use Apache Kafka in our event driven service architecture. It handles high loads of messages really well. We use Apache Spark to run analysis and machine learning.When I work with Kafka, the words of Mark van Gool, one of our data architects, always echo in my head: “ Kafka should not be used as a data store!” It is really tempting for me to do so, but most of the event topics have a small retention period. Our data strategy specifies that we should store data on S3 for further processing. Raw S3 data is not the best way of dealing with data on Spark, though. In this blog I’ll show how you can use Spark Structured Streaming to write JSON records on a Kafka topic into a Delta table.Note: This article assumes that you’re dealing with a JSON topic without a schema. It also assumes that the buckets are mounted to the file system, so we can read and write to them directly (without the need for boto3). Also: I’m using Databricks, so some parts are Databricks-specific.DesignTo make things easier to understand, I’ve made a diagram of the setup we’re trying to create. Let’s assume we have 2 topics that we need to turn into Delta tables. We have another notebook that consumes those delta tables.Each topic will get its own Delta table in its own bucket. The topics are read by parametrised jobs that[…]

    13.11.2019 | 8:21 קרא עוד...
  • Getting High Coverage Regression Tests Quickly (Part 2): Improving test coverage using approvals

    This is the second of three blog posts in which I talk about what I learnt while attending Emily Bache’s workshop ‘Getting High Coverage Regression Tests Quickly’. This was a half day workshop that took place at Test Bash Manchester on 2nd October 2019. Before reading this blog post, I’d recommend reading part 1 first. In my previous blog post, we looked at the use of Approvals to validate the results of automated tests. An approval test was used to replace a large number of asserts. The approval test takes a snapshot of the test output, and compares this with a previous snapshot. This shows the developer if anything has changed in the application. In this blog post, the approval test will be adapted so that there is an increase in the test coverage. For this exercise, I used Visual Studio with the ‘dotCover’ and ‘Resharper’ plugins. Getting more coverage With the test setup the way it was, we were already achieving 70% code coverage. The aim of this exercise was to increase this to 100%. If we look at the code, the lines which are covered in the tests are marked in green. When looking at the shopping cart class, I saw that there were 2 methods for adding items to the cart – AddItem (which only adds a single named product) and AddItemQuantity (which adds a specified quantity of the named product). So I added an additional line of code to the original test. This added a single[…]

    13.11.2019 | 2:30 קרא עוד...
  • Un-just yourself

    On Agile Testing Days 2019, Damian Synadinos gave a keynote with the topic „More Than That“. His basic message was, everybody could put more than one „label“ on herself. We’re more then testers or developers, we’re also human beings, parents, children, husbands, wifes, players, musicians… and so on. Another important advice Damian gave is „un-just yourself“. And he means that we should not make ourselves small. We’re not „just“ testers; we are testers. That resonated strongly with myself, already during that talk. Why? I think I’m using „just“ way too often. For example at work, when I’ve filed a bug, and a developer or project manager approaches me and asks: „How the heck did you spot that? That’s such a difficult thing to find!“, I used to reply „I’ve just done my job.“.  And thanks to Damian I now have a totally new perspective on that: I found that bug by doing my job, by being a tester. I’m in that field for nearly 15 years, I learned it, from the ground, attented trainings, workshops, meetups & conferences, I’m reading books & blogs, I’m practicing testing to constantly get better at it. I’m a professional software tester, that’s why I’m able to spot that difficult-to-find-bug, that a lot of other people would not have found. I’m a tester, not „just“ a tester. After that talk, during visiting other talks at the conference, I noticed how often other speakers were using „just“ in their talks, and by that diminuishing something they[…]

    13.11.2019 | 2:15 קרא עוד...

טיפים

לרשימה המלאה >>