The Messy Reality of Scale: Synthetic Data and Pre-Training — Marah Abdin & Robert McHardy, poolside
The Messy Reality of Scale: Synthetic Data and Pre-Training — Marah Abdin & Robert McHardy, poolside
Summary
Marah Abdin and Robert McHardy walk through the practical, often unglamorous engineering behind scaling small language models — the Phi and Laguna families — from a Microsoft-era enterprise release into open-weight models competitive on agentic coding. Abdin, the synthetic data lead, explains why synthetic data isn’t a replacement for organic data but a complement: organic data hides a lot of implicit rationale and structure that synthetic generation can expose and regularize. Every synthetic pipeline, in her framing, reduces to six modular components (seeds, metadata, generator, filters/validators), composable from cheap rephrasing pipelines up to expensive multi-stage, multi-agent workflows orchestrated by their in-house “Hive” infrastructure — a queue of agents with a supervisor policing generation quality and a dynamic orchestrator deciding what runs next.
McHardy then pivots to the pre-training side, framing the team’s philosophy as “trust nothing”: with models scaled to hundreds of billions of parameters across thousands of GPUs, they run parallel model replicas and continuously hash-compare their weights — any mismatch crashes the run immediately, because that invariant should never break. This is how they caught a broken GPU silently corrupting a training run (visible only as a spiky loss curve and huge gradient norms with an otherwise identical configuration), and a numerical precision bug where growing activations before the unembedding layer exceeded what BF16 accumulation could represent, flattening the loss curve until they moved the accumulation to FP32. A separate, sneakier failure came from a race condition in DeepGEMM’s open-sourced FP8 kernels, which silently corrupted about 0.5% of gradients — a genuine blind spot, since normal replica-hash checks can’t catch it in real training runs where there’s no redundant data to compare against.
All of these lessons rolled into Laguna-S, a not-yet-public 118-billion-parameter model (8B active) trained on 30 trillion tokens across 4,000 GPUs, built specifically to stress-test whether the data and numerics fixes actually held at scale. Early base-model coding evals show it beating not just poolside’s own larger Laguna-M.1 but also GLM 4.5 Air, Nemotron 3 Super, and the larger DeepSeek V4 Flash Max on coding-specific benchmarks like SWE-bench Agentless Multilingual — though it trails on general knowledge benchmarks like MMLU Pro, a gap the team attributes directly to a data tradeoff they made in favor of agentic coding strength.
Highlights
”Generate the setting, the characters, the plot — then the chapters”
“An example of this — if you wanted to generate a novel, for example — you could generate one chapter at a time, but you could also take it a little slowly: first generate the setting, the character names, the character styles, the plot, some twists, and then from there go into generating the chapters one by one. You will absolutely get a better novel.” — Marah Abdin, 6:23
Clip command
yt-dlp --download-sections "*6:23-7:14" "https://www.youtube.com/watch?v=KhYifX22yhE" --force-keyframes-at-cuts --merge-output-format mp4 -o "multi-stage-novel-pipeline.mp4"
”We don’t trust anything”
“We know there’s an invariant: the weights should always be the same across all of these replicas. You can calculate a hash over the weights and you know that should always be the same across all replicas. So we do that in training and periodically compare them. If all of these hashes are identical then we know we can continue training. If they’re not identical, we know something has gone seriously wrong because that should never happen, and we crash the training.” — Robert McHardy, 10:00
Clip command
yt-dlp --download-sections "*10:00-10:34" "https://www.youtube.com/watch?v=KhYifX22yhE" --force-keyframes-at-cuts --merge-output-format mp4 -o "trust-nothing-hash-check.mp4"
”A broken GPU caused silent data corruption”
“There’s actually no difference in terms of model configuration, training data, training implementation between these runs. They’re exactly the same run, just in one of them we got unlucky and we had a broken GPU included. That broken GPU caused silent data corruption and therefore made the training behave the way it did.” — Robert McHardy, 10:34
Clip command
yt-dlp --download-sections "*10:34-11:04" "https://www.youtube.com/watch?v=KhYifX22yhE" --force-keyframes-at-cuts --merge-output-format mp4 -o "broken-gpu-silent-corruption.mp4"
”Not enough numerical precision to learn anymore”
“That accumulation was performed in BF16 by default and because of the growing scale that we observed in the activations, there wasn’t enough numerical precision available anymore to do this accurately, and hence the model just couldn’t learn anymore… we moved that accumulation into FP32, and then from there on the model started converging again.” — Robert McHardy, 11:56
Clip command
yt-dlp --download-sections "*11:56-12:43" "https://www.youtube.com/watch?v=KhYifX22yhE" --force-keyframes-at-cuts --merge-output-format mp4 -o "bf16-to-fp32-fix.mp4"
”0.5% of the gradient gets silently corrupted”
“We had a race condition because we added FP8 training based on DeepGem FP8 kernels that also got open sourced… In our case, we noticed about 0.5% of the gradient gets silently corrupted, essentially replaced by random values… it’s also an interesting point because it’s a blind spot in the hash checks.” — Robert McHardy, 13:59
Clip command
yt-dlp --download-sections "*13:59-15:00" "https://www.youtube.com/watch?v=KhYifX22yhE" --force-keyframes-at-cuts --merge-output-format mp4 -o "fp8-race-condition.mp4"
Key Points
- Speaker context (0:12) - Marah Abdin (synthetic data lead) and Robert McHardy, both at poolside (formerly Microsoft’s Phi team), discuss scaling small models toward open-weight releases.
- From enterprise to open weights (0:12) - Phi-3 Mini and Phi-3 Small released on Hugging Face; a tech report details what changed between the 4K and 128K context versions.
- Three data-side fixes (1:22) - An auto-mixer for cheap sweeps, rethought web-data sampling for higher recall, and heavier reliance on synthetic data.
- Synthetic data complements, not replaces (1:22) - Organic data hides implicit rationale, planning, and structure; synthetic generation exposes and regularizes it. Phi-3 Mini’s pre-training mix was 13% synthetic; the corpus has since grown to 6 trillion tokens.
- Token uniqueness problem (3:03) - Early quality-over-quantity focus caused non-optimal repetition once training budgets scaled, saturating the model early; rephrasing (including specialized code-to-text and STEM pipelines) fixed it.
- Six components of any synthetic pipeline (5:00) - Seeds, metadata, secondary inputs, a generator function (agent or LLM with a prompt template), and supplementary filters/validators.
- Break the task down (6:00) - If a task is too hard for the teacher model, it falls into bias and loses correctness/diversity — simplify and decompose instead.
- Four pipeline “shapes” (6:23) - Formatting/rewriting (rephrasing), multi-stage pipelines, cross-domain porting (e.g. math problems to code), and multi-turn roles (agent-to-agent chats, judge/evolver loops).
- Hive infrastructure (7:54) - Configurable queue of agents with defined prompts/params/models, an orchestrator that polices hierarchy between generating LLMs, and a supervisor with a global view.
- “Trust nothing” philosophy (9:33) - McHardy’s team treats data quality and training-codebase correctness as equally load-bearing; bad data or a buggy codebase both sink the model.
- Model replica hash-checking (10:00) - Periodically hash and compare weights across data-parallel replicas; any mismatch crashes training immediately.
- Broken GPUs at scale (10:34) - A single faulty GPU silently corrupted data and produced a spiky loss curve with huge gradient norms, in an otherwise identical run.
- Exploding gradients from BF16 accumulation (11:30) - Growing activations before the LM head exceeded BF16’s numerical precision during tensor-parallel unembedding accumulation, flattening convergence around 50,000 steps.
- FP32 fix for the accumulation (12:00) - Moving the accumulation to FP32 from the affected checkpoint restored convergence and a decreasing gradient norm.
- From Exa-1 to Exa-2 (12:43) - All the data-repetition and numerical fixes rolled into Exa-2, a 33B-parameter open-weight model competitive for its size, focused on agentic coding.
- Laguna-S: the scale test (12:43) - 118B total / 8B active parameters, trained on 30 trillion tokens across 4,000 GPUs, built to verify the fixes held at much larger scale.
- FP8 race condition (13:59) - DeepGEMM FP8 kernels caused illegal memory accesses and NaN gradients from a race condition; ~0.5% of gradients silently corrupted. A fix is in an unmerged PR.
- A blind spot in the hash checks (13:59) - Real training runs have no redundant replica to compare forward/backward passes against, so race conditions there go undetected; the team is building a dry-run hash checker for this.
- Laguna-S coding benchmarks (15:29) - Beats Laguna-M.1 (larger), GLM 4.5 Air, Nemotron 3 Super, and DeepSeek V4 Flash Max on MultiPL-E, LiveCodeBench, and BigCodeBench.
- Strong on SWE-bench Agentless Multilingual (15:57) - Used as an agentic-performance proxy during pre-training; Laguna-S leads all compared models here.
- Known tradeoff: general knowledge (16:24) - Trails Nemotron and DeepSeek on MMLU Pro, a gap attributed to a deliberate data choice favoring agentic coding strength.
Mentions
Companies
- poolside ([description]) - Abdin and McHardy’s employer, building the Exa/Laguna model families for agentic coding.
- Microsoft (0:12) - Abdin’s stated affiliation at the time of the Phi-3 releases discussed at the top of the talk.
- Hugging Face (0:12) - Platform where Phi-3 Mini and Phi-3 Small were released as open weights.
- DeepGEMM / DeepSeek (13:59) - Source of the open-sourced FP8 kernels that caused the race-condition bug.
Products & Technologies
- Phi-3 Mini / Phi-3 Small (0:12) - Microsoft-era open-weight small models discussed as the starting point.
- Hive (7:54) - poolside’s configurable synthetic-data generation infrastructure (agent queues, orchestrator, supervisor).
- Exa-1 / Exa-2 (12:43) - poolside’s 33B-parameter open-weight coding model generation, testbed for the fixes described in the talk.
- Laguna M.1 / Laguna-S (9:33) - poolside’s larger model family; Laguna-S is a 118B (8B active) preview model trained at much greater scale.
- GLM 4.5 Air (15:29) - Comparison model Laguna-S outperforms on coding benchmarks.
- Nemotron 3 Super (15:29) - Comparison model; outperforms Laguna-S on MMLU Pro but trails on coding.
- DeepSeek V4 Flash Max (15:29) - Larger comparison model that Laguna-S beats on coding benchmarks.
- MultiPL-E / LiveCodeBench / BigCodeBench (15:29) - Coding benchmarks used to evaluate Laguna-S.
- SWE-bench Agentless Multilingual (15:57) - Used as a proxy for agentic performance during pre-training.
- MMLU Pro (16:24) - General-knowledge benchmark where Laguna-S trails larger competitors.
People
- Marah Abdin (0:12) - Speaker; synthetic data lead at poolside.
- Robert McHardy (1:22) - Speaker; leads architecture and distributed-training work at poolside.
Surprising Quotes
“If you’ve got data that sucks you can’t train a good model, if you’ve got a training codebase that sucks you also can’t.” — Robert McHardy, 9:05
“The rule of thumb is if a task is too hard for your model then your model will start to fall in its bias, lose correctness, lose diversity. So break down the task, make it simpler.” — Marah Abdin, 6:00
“First example here of shit that happens at scale are broken GPUs.” — Robert McHardy, 10:34
“It’s not the strongest model in the world… this mainly comes down due to data, right? It’s a data gap that we could plug if we wanted to.” — Robert McHardy, 16:24
Transcript
Marah Abdin: 0:12 Hi everybody, thanks for coming to our talk. My name is Marah Abdin, I’m from Microsoft, I’m the synthetic lead for our data team. And today my colleague Robert and I will be talking a little bit about some of the challenges that we’ve seen as we scale our models over here, particularly if you haven’t heard, we’ve switched recently from releasing our models towards enterprise to also releasing towards everybody. We’ve actually put out two open-weight models on Hugging Face a few weeks ago: Phi-3 Mini and Phi-3 Small. We also put out a tech report which has a ton of detail if you’re interested. As you can see here by the Phi-3 Mini 4K and 128K, this actually is because we have switched out quite a few things between those two models and so a big flavor of this talk is going to be about kind of how did we transition from one to two. And in fact, we’ve continued to do so and now we actually have a newer version and Robert will give a sneak peek about soon-to-be-released model.
Robert McHardy: 1:22 Okay. So, I will be particularly talking about synthetic data part of things. So, there’s three things that we did on the data side to kind of resolve some of the issues that we’ve seen with scale. One is that we implemented an auto-mixer that basically just gives us the chance to do a cheaper sweep on clusters of our sets before moving on to more expensive experiments. And then we improved — we just rethought our sampling of web data for higher recall. And then the third one is that we relied a lot more on synthetic data in a few forms, which I will go into. Okay, so before we kind of go into what does that mean and what have we done etc. Why would we kind of — it’s sometimes it’s fair at least to ask why synthetic data. And the thing is that at least at Microsoft we don’t see it as a way to replace organic data, I don’t see it as so in the current state of the world at least, but it is a way to kind of complement it. And the thing is that organic data has a lot in it that is basically kind of implicitly hidden. A lot of things that could teach the model are not very presented in the most optimal way sometimes. And so synthetic data gives us a track to extract some of these features and project them on some new planes. And this is how we get to expose implicit rationale, implicit planning, implicit structure and a way for us to fill gaps and regularize not only how we present the tokens but also how we are teaching the model. For Phi-3 Mini in particular, we settled on 13% of the mix, this is only pre-training stages before post-training. And since then we’ve just been continuously generating more data in a bunch of directions, now we have a 6 trillion token corpus that’s…
Marah Abdin: 3:00 is growing.
Robert McHardy: 3:01 Okay.
Marah Abdin: 3:03 So yeah, so kind of what I just said is that we saw some limitations switching from Phi-1.5 to .1 or .2 models. And so one of those things is that we basically started on data, and this is really not a crazy kind of problem. We very intuitively started from a place on a smaller scale where we were basically focusing on quality versus quantity, maybe a little too much, because eventually when we started scaling our models, we had to scale our training budget, and with that came some limitations because we started hitting repetition, like non-optimal repetition, on some of our higher quality data, which saturated the model a little too early. So, one of the ways that we’ve, particularly for this token uniqueness problem, we relied on, which is a very common form of synthetic data, rephrasing, which you might have heard of in Bing on Web, for example, it’s become pretty trendy these days. And you can see here that this is an ablation result, so take the numbers with a grain of salt, but what persists pretty consistently is the diff between using the orange would be just the seeds with repetition, and then the green would be replacing some of those repeated tokens with higher, like with multi-mode rephrase, or at least reducing the repetition. And so for rephrasing particular, we did to the, you know, what everyone’s doing with the generic kind of multi-mode very scalable pipeline, but we also took it a step far and we did two other specialized pipelines, one for to go from raw code to code and text, and one to go specifically for STEM data, just because this is a very cheap, scalable pipeline. So it kind of, you kind of have to rely very heavily on the seed and we push a little further on that for the STEM documents.
Robert McHardy: 4:59 Okay.
Marah Abdin: 5:00 So, okay, so if you kind of think of everything in a modular way, you can think of every synthetic data pipeline is composed of the same six components. And so you have your seeds, your primary inputs, your metadata, your secondary inputs, your generator function, which can be an agent with tools or an LLM with some prompt template, and then some supplementary functions like filters and validators and so on. And really you can compose just about all pipelines from very simple to very expensive pipelines like this. And on that note, we’ve covered quite a bit of a wide scope on the axis of complexity, and you can think of it if at one end you have the cheap scalable pipelines that have used smaller models and can get away with it because they’re seed heavy, example rephrasing, and then on the other end, you have more complex pipelines with a little more orchestration in the workflows. This is reserved when we’re building on something that’s worth it, educational data…
Marah Abdin: 6:00 but really it’s this is how we’re not blocked or limited by whatever teacher model can do and this is how we can be ambitious in our synthetic data because the rule of thumb is if a task is too hard for your model then your model will start to fall in its bias, lose correctness, lose diversity. So break down the task, make it simpler. And I will give some examples of kind of shapes rather than just something more concrete.
Marah Abdin: 6:23 To talk about how did we use this modularity, one shape is formatting/rewriting, it’s just rephrasing we already talked about this. Multi-stage pipelines and multi-stage workflows, basically this is what I also just said: you take a step and you break it down into multiple steps, you can aggregate the process and slowly build up the generation. An example of this — if you wanted to generate a novel, for example — it’s you could generate one chapter at a time, but you could also, you know, take it a little slowly: first, like, generate — I don’t know — the setting, the character names, the character styles, the plot, some twists — and then from there go into generating the chapters one by one. You will absolutely get a better novel.
Marah Abdin: 7:14 Third is cross domain porting, which really is just moving from one mode to another. Example would be changing, like, translating code. Another example would be something we did which is take our math problems and convert them to code.
Marah Abdin: 7:29 The last one is multi-turn roles. What do I mean by that? All I mean is that instead of having a very linear or even nonlinear kind of view of things, you have more of an iteration, and this encapsulates pretty much everything. And by that, an example of that would be multi-turn chats when you have two agents talking to each other, or a task evolution pipeline where you have a judge and an evolver going back and forth for some K amount of times, and so on.
Marah Abdin: 7:54 So lastly, handing off to Robert, I do want to kind of just mention that because of the modularity of the way we think about this, we can implement infrastructure that’s pretty configurable. So this is how we present Hive. Hive is basically a way for us to easily construct generations where now you have a queue of agents that you define, each one has its prompt, its parameters, its model, etc., inputs, outputs, but it also has when you can configure — one enters the queue, when it exits, how much frequency it comes in — and then we have orchestration in the middle between agents, and what these orchestrators are really useful because they give you more flexibility in kind of presenting a hierarchy between the LLMs that are generating. It’s this is how you police them basically, but it also gives them some form of creativity and dynamically change instructions per the next agent, or choose which agent goes next, which agent skipped, and so on. And lastly, you have the supervisor, which basically someone who polices the orchestrator and has more of a global view. Cool, okay, that’s it from me on synthetic data, hope you learned something interesting, handing over to Robert for reinforcement learning…
Marah Abdin: 9:00 …pre-training aspects.
Robert McHardy: 9:03 Alright, thank you Marah.
Robert McHardy: 9:05 Cuz I will talk a little bit more on the actual pre-training side rather than just data. I liked in the previous talk the speaker made a point that we should treat different data mixes holistically, different training stages. I want to make the same point that we should treat data and implementation of your training codebase, correctness of it and so on also holistically. If you’ve got data that sucks you can’t train a good model, if you’ve got a training codebase that sucks you also can’t.
Robert McHardy: 9:33 So I specifically focus on architecture work on and distributed training and so on. And the way we look at things in my team is we don’t trust anything. There’s so many things that can go wrong when you scale models to billions of parameters to hundreds of billions of parameters, train on thousands of GPUs and so on. And I want to show you some of the learnings that we got from training Laguna M.1 and some of the surprising things that happen at scale.
Robert McHardy: 10:00 So one thing we do is we’ve got these model replica hashtags, so essentially when we train a model we’ve got multiple replicas of the same model, right? Distributed data parallel. And we know there’s an invariant: the weights should always be the same across all of these replicas. That’s something you can verify, right? You can calculate a hash over the weights and you know that should always be the same across all replicas. So we do that in training and periodically compare them. If all of these hashes are identical then we know we can continue training. If they’re not identical, we know something has gone seriously wrong because that should never happen, and we crash the training.
Robert McHardy: 10:34 And I will give you some examples now of things that we’ve not shared before publicly like this, so I hope they’re interesting. So first example here of shit that happens at scale are broken GPUs. On the left hand side we’ve got two loss curves and on the right hand side the corresponding gradient norms that we observed during training. And you can see that these loss curves look quite different, right? Like the purple one has got quite some bumps, looks a bit spiky, the gradient norms are huge for that run. And there’s actually no difference in terms of model configuration, training data, training implementation between these runs. They’re exactly the same run, just in one of them we got unlucky and we had a broken GPU included. That broken GPU caused silent data corruption and therefore made the training behave the way it did. And that is one of those cases that you can catch with these hashtags because you know this computation should be the same across all replicas, but it wasn’t.
Robert McHardy: 11:30 Which brings me to the next instance of shit that happens at scale, and in this case exploding gradients. Again we’re looking at two different loss curves and the corresponding gradient norm curves. The purple run is our initial training run for Laguna M.1, we’re a bit further into training here around 50,000 steps or so, and you can see it stops converging, right? Like it just flattens out. And the reason here was that during training the activations grew and grew right before the LM head, the unembedding, and we have to perform some sort of accumulation here because we use tensor parallel for the unembedding.
Robert McHardy: 12:00 And that accumulation was performed in BF16 by default and because of the growing scale that we observed in the activations, there wasn’t enough numerical precision available anymore to do this accurately, and hence the model just couldn’t learn anymore. And this is also a very dramatic point for this to happen because it from there on, it really back propagates into the full model trunk. The orange curve is essentially just adding a fix on that, so we took the checkpoint from the purple curve, we moved that accumulation into FP32, and then from there on the model started converging again. The gradient norm, as you can see, actually started decreasing. Before then we had an increasing trend, and this is also something you can only observe at scale, and that will break your model if you’re not careful about it.
Robert McHardy: 12:43 So, as Marah said, we took all of these insights on data, on numerics and so on, and we turned them from Exa-1 into Exa-2. That’s why we say it’s a newer generation model. This included increasing diversity, reducing data repetitions, all of these numerical things I just mentioned, and just adding more observability and checks on that side, as well as generally optimizing the training and the architecture. And Exa-2, if you look at it, its weights, right, so you can download and use it for free. It’s one of the most competitive models for its size and for coding specifically, that’s what we focus on, agentic coding. So, pretty happy with it. However, you can say that the model with 33 billion parameters is pretty small, like you wouldn’t probably observe any issues anyways training it. So, what was important to us was to scale this, right? And this is where Lagon-S comes in, this model is not public yet, so this is a preview. As I said, we treated Exa as a testbed in a sense, and with Lagon-S we scaled this to a model that’s 118 billion total parameters and 8B active parameters. Again, we trained it on 30 trillion tokens on 4,000 GPUs. So, the scale was sufficient to not only test whether all the improvements we made on the data and architecture side hold, but also if any of these numerical issues come up again.
Robert McHardy: 13:59 And, of course, something happened, and in this case, it doesn’t actually have anything to do with scale, so it was just unfortunate. In this case, we had a race condition because we added FP8 training based on DeepGem FP8 kernels that also got open sourced. We noticed these because we hit illegal memory accesses as well as NaNs in the gradients, which after a while of debugging we traced back to those kernels. There’s also an unobservable effect that you wouldn’t know about if you don’t know that there’s an issue. In our case, we noticed about 0.5% of the gradient gets silently corrupted, essentially replaced by random values. We do have a fix available that’s in a PR right now. It’s not been merged into DeepGem yet, but it’s public on that QR code if anyone is interested. And it’s also an interesting point because it’s a blind spot in the hash checks. In real training runs, you don’t have any redundancy where you have the same model weights and the same data, so you can never check if forward and backward actually behave the same across different model replicas, so you can also never check if there’s a race condition in that. That’s something that we’re working on right now to essentially have a hash checker…
Robert McHardy: 15:00 …that can also do that as a at least as a dry run.
Robert McHardy: 15:03 And I want to end on some early results from this new model and demonstrating how it performs against some open weight models and also against our previous models.
Robert McHardy: 15:14 So first I want to caveat this with these are base model evals, right? They are partly indicative of how the final model will look, but also not perfectly, right? There’s still post-training happening, not all of these will translate one-to-one to the final model.
Robert McHardy: 15:29 But if we look at them, specifically on the coding part of the evals, so for instance Multiple-E, LiveCodeBench, BigCodeBench, Laguna S is not only stronger than XS.2, which is our previous smaller model that performed very well, but also than a much larger M.1. And it’s also much better than GLM 4.5R, which is admittedly a bit older, Nemotron 3 Super, which is quite recent, and then DeepSeek V4 Flash Max, which is quite recent and a fair bit larger.
Robert McHardy: 15:57 We can see it’s competitive on Big Bench Hard, for instance, it doesn’t achieve the top eval result compared to these models, but is quite close. We also see it’s quite close on Eval Plus and, quite importantly for us, it does very well on SWE-bench Agentless Multilingual, which we use to sort of proxy agentic performance during pre-training. And in that case, it performs much better than all the other or all other models we tested here.
Robert McHardy: 16:24 I also want to point out that of course there are, like it’s not the strongest model in the world, right? Like for instance MMLU Pro knowledge benchmark is something we don’t care about that much compared to coding because we want to build the strongest agentic coding models. So here, like compared to Nemotron and DeepSeek, we have to say that they perform much better and this mainly comes down due to data, right? It’s a data gap that we could plug if we wanted to.
Robert McHardy: 16:46 But I think the point is all of the things we’ve found before were included in a recipe, the recipe held, it scaled, and we will continue scaling it from here. So this model will also be available sometime in the future relatively soon, again open weights so all of you can download and use it for free.
Robert McHardy: 17:01 And with that, I want to thank everyone for attending our talk. I added also a link to our careers page and our Twitter page if you want to check it out. And yeah, thank you very much.
