Kristin Tynski: Yeah, I mean, and even includes things like, cuz then the training corpus had like, audience reactions in it. You can see like, the fourth line, has booing him reacting to the booing.

Jordan Choo: That's, that's really impressive.

Kristin Tynski: Yeah, I honestly, if I showed this to either of you guys, and told you that this was part of Trump's speech yesterday. Like, I don't think you'd come back to me and say, I think this was written by an AI. No, not at all. So do you plan on rerunning your experiment with the marketing blog that doesn't exist with control? And she's probably not because control is, is the much larger model and it takes a lot longer to to create text. So it would probably take a few weeks to generate a few hundred articles. And it would cost probably a few hundred dollars, at least. I don't know. I mean, it's it's tempting to see if if there is any improvement on this versus the rover model that I mean, this marketing blog does not exist from Yeah, maybe in the future, I don't know, I, I have some misgivings about doing a lot more of, you know, putting this kind of texts onto the internet, like outside of the goal of just trying to publicize this. The scariness of this technology. I've actually already seen that there are a couple companies that are you using GPT to not Grover, but somebody that's using a, one of the larger GPT to model cell automated text, or automated articles, essentially. So that I mean, it's already beginning, you know, there are already a couple services that provide that, which to me is is kind of crazy. And I think they sell the articles for like, 50 cents apiece, like 500 word articles. He's I mean, it could be it could be creating millions of day.

Noah Learner: Wow.

Jordan Choo: scary stuff.

Kristin Tynski: Yeah, yeah.

Noah Learner: So what lessons can you teach us about scaling in a an agency? You've done it twice?

Kristin Tynski: Yeah. So my first agency, I started with my brother and my brother in law and 2007, right out of college. It was basically, it was a content marketing agency. But the focus was really on infographic creation, when that was kind of the thing that really, really works well for link building. And after operating that, for three or four years, we decided to sell it to a company called blue glass in Tampa, and become part of their team that didn't really end up working out the way that we thought it would or should. So we ended up leaving there after about a year, and taking the lessons that we learned through that process and deciding to start a new agency. And that that's the agency that we started is fractal. So fractal is about six or seven years old now. And we are a content marketing agency, our focuses on data journalism driven link building, so trying to generate press through the vehicle of data driven storytelling. And our agency is essentially made up of content creators and content promoters. And the way that we at least in my opinion, the two biggest factors in helping us grow or the quality of our work differentiates us or processes differentiates us because we do more than just content creation, we also promote that content, our results differentiate us. And then we do we do a lot to generate inbound lead flow. And that really comes down to doing things like this marketing blog does not exist, which are, at least in our opinion, kind of thought leadership pieces for, for the industry, trying to start conversations, and then that also doing a lot of work exploring the aspects of our business, and that really matter to success. So what goes into creating content that can drive that can really get news backups? What are publishers interested in? How do you pitch publishers properly? How do you leverage data, data science machine learning? to to really come up with new newsworthy stories that other people haven't done before? And then how do you pair that with, with digital PR to effectively get it picked up by the press? Then how do you support that with, with additional outreach and additional syndication support to make sure that you can get like the maximum value possible out of the work that you've done? And the promotion is what it feels to me like, there's so many different ways of thinking to do. There's so many different types of brains to execute that. Well. Howdy, how did you guys all split up the roles? Like how does how does your executive team split up that because writing is so much different than data science? You know what I mean?

Kristin Tynski: Yeah, I would say our creative team has to had to become proficient on storytelling and also data analysis. That's, that's really a role of a data journalist. And that's, that's how we think of our content creators here. But their main purpose is to tell a story, but they're doing it through data. And there's been a learning curve to that. But we've been working on that, for know, pretty much the entire time, we've been working on fractal and we've doubled a team that is has become experienced in doing it. And through iteration, over time, we've learned what works well. And what doesn't work well. What sorts of projects require more effort than they're actually worth, which ones will have a high ROI for the amount of effort required, and what sorts of topics and what sorts of data driven content executions seem to seem to resonate Well, with the people that we pitch. So there's a lot that's gone into it over the long term, and getting it right and improving, and being able to do something that can scale and offer similar results across a lot of different clients. And in terms of our team, I mean, I'm really lucky that it's a family business. So the people that own it are me, my brother, my brother in law and my partner. And that has made things a lot easier, because everyone's really aligned all the time, you know, we're all working toward the same goal. And we y'all have some familial connection, but with that son, can also be hard working with family. And so it's really important to, like divide and conquer, and for each, each person that you know, each partner and fractal to, to really have a focus on a different area of the business and for their not be a huge amount of overlap between us, when we come together, when we have to make decisions that affects the entire agency. And, and set goals and things like that. But I like I'm not the CEO, I, I run the creative team at fractal. And my brother runs sales, my brother in law, Nick does is the CEO. So he, he does all the CEO responsibilities. And then my partner, Kelsey, she's she manages the outreach team, and the growth team. But we all contribute to other pieces too. So I, I like to contribute a lot to like the thought leadership and outreach because it allows me to kind of like stretch my creative muscles and explore new methodologies that we maybe wouldn't do for clients. But because we have more time to do things for ourselves, I can I can do more experimental things and try and find answers that aren't really easy to come by, that we're interested in as an agency, and that we think the greater SEO and content marketing industry would also be interested in

Noah Learner: what parts of the business have you guys found easiest and hardest to manage? Or scale?

Kristin Tynski: I don't think any of them have been easy to scale, really. Because the type of work that we do is, is content creation. You know, each individual project probably requires 40 plus hours of work from our content creators, and then, you know, maybe 30 to 40 hours of working in pitching. Yeah, it's it's I mean, it's a lot because it's one to one pitching, it's, it's, it's trying to find the right home for something that we consider, hopefully about the publishers were pitching considered interesting and newsworthy. And so you have to approach it in a in like a very high touch way. So there, there are ways to that we, you know, we've done a lot of work improving internal processes to try and remove bottlenecks, and to try and maintain quality assurance. And so there are efficiencies that can be gained process wise internally, but in terms of like the work itself, and there are limitations to like how low you can go, I think, just because of the nature of the work is is pretty time intensive.

Noah Learner: Hi, original question. I'm sorry.

Noah Learner: Yeah, no, that that's, that's pretty amazing. I was, I spent some time on your LinkedIn profile before the call, and I, I loved how you had those two links at the top of your profile how you link right into the deck. I thought that was really cool. And one of the things that I remembered on the deck was that you average something like 90 backlinks per per project. And I'm just doing the math, if you're doing 30 hours of pitching, and you're getting 90 links, it's pretty fascinating. That's so much work. I mean, content promotion is the thing that I feel like in my small agency, I just don't have enough time for oftentimes in it. And I have, I can dedicate some time per client per month, but not enough. And I know that that's where the rubber meets the road. And we've automated so many processes so that we'd have more time for things like content promotion. What have you guys been able to automate? Sorry to ramble at you. But what have you guys been able to automate that you're really proud of.

Kristin Tynski: In terms of automation, it really has mostly to do with with like workflow. And so we we leverage a lot of like off the shelf tools, so like our project management software as a service. And that's, that's worked great for us. We use slack internally. And then we, we use a tool called clip folio to essentially monitor our productivity and the quality of our work. So we've built dashboards for each segment of our business to understand the promotions team and the production team independently. So we have metrics on how many pitches are coming out each day, what the success rates of those pitches are, what the placement rates are, what the rejection rates are, what the response rates are, we can look at that granular the each individual MRI or Meteor Media Relations associated, as we call them, we can look at it by account team, we can look at it by client. And then we from that we can extrapolate and understand benchmarks across the entire agency and across the cow groups and across clients. And that helps us set expectations for those team members and measure their work against those expectations. On the content side, we have something very similar, it's just the metrics that we use to evaluate them and to evaluate our our efficacy are based on the quality of the content and the speed at which it can be produced. So we set benchmarks for how how long an individual project should take what the averages are, the median is for an individual team member across a quarter or across half a year. What that looks like by account team, what it looks like my client, and then lots of other metrics that are about the quality of the work. So we do to internal peer reviews per project. So we do want an early stage and want to sort of like a final draft stage. and I both of those stages we are recording, our senior creatives are scoring and recording the quality of those pieces of content. And then that's part of part of one of the metrics that we use to evaluate the creatives that produce them, like on a quarterly or half year basis. So we've automated like the the analysis part a lot. And I think that's helped us to understand where we need to work on things and where things are working well and where things have improved. It gives us insights into how we can adjust internal processes to become more efficient and, and better quality and create better quality work. And in terms of automating the process of data journalism, or content creation using data, there's a ton of individuals specific things. So like, you know, dozens and dozens of scripts that have been written by our development team to help automate parts of the data collection process for, for a lot of different specific individual cases. And some that are extensible. So like we've done a ton of work with, like the fatal accident reporting system, which is a government data set, that gives a huge amount of detail on fatal accidents. So we've use this data set for a variety of clients, and like the car insurance and car sales space. And so we put a good amount of time into decoding that data set and putting it in formats that we can easily analyze. So a lot of times there's there's a lot of hidden time and and money spent in that data acquisition and data cleaning portion of the content production process. And you don't want to have to do that same thing over and over again. So where there are opportunities for us to, like simplified either parts of those process data cleaning or data acquisition, we do that. And then we can reuse those pieces of code later on. other team members can use them when they're doing similar projects, or even completely different projects to rely on the same data sets. So we do a lot of that. The other part is just trying to help our team level up. So we want, you know, in the next year or so we want our entire creative team to learn the basics of Python. Just because it can massively simplify some of those early stages in the in the data acquisition and data analysis process are creatives can themselves scrape data from the internet, or can themselves transform and clean data using pandas and Python, it can save, you know, 20% 30% of the time that they would have spent, you know, trying to pull it in and out of Excel or these other tools to clean data that are, are not as extensive Well, not as fast and just more difficult to use generally, than if you you know taught yourself or learned the basics of some of these, these Python libraries that will help you clean and manipulate data much better and faster. So

Noah Learner: I thought amazing, I'm, I'm I feel like total newbie, the thing that's frustrating about it is you'll spend time in Git Hub, you'll find a repo that you want to work with, you'll then try and pull it in. And then you try and run it in the first two hours is just like dealing with broken dependencies. It just drives me flipping nuts. And it's just

Kristin Tynski: I I know exactly how you feel I'm I'm a relative Python newbie to and yeah, I mean, like setting up these these transformer models is like setting up back the Salesforce control and it was like a giant pain in the ass. Even though it was like decently well documented, like, there's, there's just a lot of ways that you can go wrong that especially if you're relatively new to it, you don't have the intuitive understanding of why or what might be wrong. And so yeah, it can just be like a very clunky, frustrating process. But I found that when I have something that I'm really interested in seeing work, I'll have like the dedication and fortitude to see it through. And through that process, I'll learn a lot more. So like for me part of learning and getting better at Python is picking like these, these pet projects that I find fascinating. And then just just brute forcing it until I can get to where I want to be.

Noah Learner: Whoo. You know, I think I'm going to acquire that as my new superpower. I've told people that grit is and not giving up. But I think it's more brute force.

Kristin Tynski: The key is to be you have to be interested in the thing that you're going for, like I would have given up so long ago, if I wasn't just totally fascinated with these new technologies, and you know, wanting to gain the superpower that they provide myself. I just, that's a compelling reason to learn something from me.

Noah Learner: Which which lessons have you learned along the way do doing all this? This kind of like AI ml stuff? What lessons have you learned? Like, what? What expensive mistakes? Can you share?

Kristin Tynski: Um, don't don't accidentally be an expensive, you know, multi GPU virtual machine on running, running on Amazon Web Services.

Noah Learner: Oh, yeah, I learned that lesson too.

Kristin Tynski: So let's see do like, actually, all night last night, because you have to like, you click, you can click stop. And then it says, Are you sure you want to stop this machine? And I thought I had clicked it. But apparently I hadn't. And I found it was running this morning. So I think I spent like $75 overnight.

Noah Learner: Oh, I was gonna ask you if you were over under 200. Cuz that was that was my that was my expensive lesson.

Kristin Tynski: Well, my brother did it too. And his lesson was much more expensive than that. And so I feel as bad going to come back and tell him that he made a much bigger mistakes. Yeah, that's that's a tough lesson to learn. Although I think Amazon and Google Cloud when that happens.

Noah Learner: And what can you share with us? In terms of like, stepping back and talking about digital marketing agency life? Can you share with us share with us your off the shelf tools besides clip folio and Asana?

Kristin Tynski: Um, I mean, for SEO, we use sem rush and H refs mostly, like social content, data? Buzz Sumo. For outreach management, we use buzz stream. And then a lot of other various like small things for like, other tasks, but like, those are the main ones that we use internally?

Noah Learner: And are you guys building lots of solutions internally with whatever language like do you guys What? What language? Do you guys lean on the most besides Python internally, to kind of tie the eyes together, etc?

Kristin Tynski: I mean, it's almost entirely Python, to be honest.

Noah Learner: Okay. Cool.

Kristin Tynski: Yeah, just because we do so much data scraping and data cleaning and data manipulation. I mean, you can do in a lot of languages, but the off the shelf stuff in the document for Python is, in my opinion, better than most other languages, so and then also pythons using machine learning. So frequently, you know, it's kind of the standard there. So, to us, we kind of double down on Python.

Noah Learner: Can you share how you scrape?

Kristin Tynski: Um, I mean, it depends on what we're scraping. There are, there are a lot of scraping libraries that you can use, we use one called newspaper, which works really, really well. And there's Beautiful Soup as well. I don't know if at all, I think different ones were better for different use cases. Like in some, I think in some instances, there are actually like built in, like web tools that you can add as like a browser extension to do web scraping, that can be easier than writing a super complex custom code to like, go through a bunch of, you know, segments of each GMO. Maybe if you were super professional writing Python code, you could do it faster than you could with some of these, these web plugin or browser plugin tools. It just really depends on the case in the site. So like we did a project, actually, for one of our We own a couple of websites that we do content marketing, or just ourselves to drive some link, secondary revenue. And we own one called lawsuit.org. And so we're trying to build it up. We're doing some link building for right now. So I, I recently did a project that was scraping public records, the public records database and Palm Beach County, for traffic citations. And within the public records, if you look at an individual record, the the police officers name and badge number that all the person over and give them the citation, we have this interesting idea of scraping that entire set of public records for the county and then doing lack of bulk data analysis of the individual police officers and their like their rates of citations based on like race or based on gender based on citation type, or based on location, and see if we could find like, dirty cops in the data or racist cops in the data. And we found some interesting things actually, that we're going to be releasing that soon. But the most difficult part of that process was in writing the scraper because it had a bunch of things to try and prevent scraping.

So like Captcha, like Google's Click here if you're not a robot CAPTCHA. So in addition to actually writing the script to navigate through the hierarchy, and click through the hierarchy, and then download the information, once it got to the right place, we also had to write some code to to handle the captcha solving and a few other small things. So each individual case is super unique, and often requires, you know, like a more technical scraping solution than you can get with like, browser plugin.

Unknown Speaker: That's amazing.

Noah Learner: So do you cackle when it when it runs?

Kristin Tynski: I mean, I was pretty happy when we went to find when we finally got to work, but it took one of our developers like three or four days to finish the code to do it. Well,

Noah Learner: that's insane.

Kristin Tynski: Yeah, it what's actually interesting is that it's there have been some court cases that have set a precedent that say, it's completely legal to fully scrape government data repositories, public government data request repositories, even if they use, like anti scraping techniques to try and prevent you from doing it. So although we had to be a little bit subversive with that approach, it's a completely legal thing to do. And actually, in my opinion, a good thing to do, that data should be accessible to a wider audience of people it shouldn't be. It shouldn't be hidden behind a whole lot of unnecessary navigation, you should just be able to get it all so that we can, as citizens, as David journalists, as content marketers, investigate important government information that affects us.

Noah Learner: That's amazing. Um, I feel out of my depth a little bit with all the stuff that you're doing, like a lot of the the automation that we do help help agencies run smoother by automating individual processes, you know, whether it's onboarding, whether it's task management stuff, I had not really thought about how to automate all the content marketing things. I build tools internally that auto generate HTML output. So you just you put in configuration variables, and it outputs HTML code, and away. It's formatted according to CSS output that we anticipate. And I feel like a rocket scientist when I get that to work, and I feel, and I know my team appreciates it, because they save a lot of time. But the stuff that you're doing is so next level, it's just it's kind of mind boggling.

Kristin Tynski: Well, thank you for saying that. But I think it can be easy to get caught up and being proud about like some technical solution to something you wanted to do, and and get a little bit lost in like, what the ultimate value of that work is. And sometimes it's like the simplest, most straightforward solutions that are co driven or not, that end up being the biggest time savers. So I think it's important to focus on ways that you can automate your business and, and utilize new technologies to help you do that. But not to get so focused on the fact that, you know, you're going to leverage this cool new technology that you don't think about, maybe there's an easier, more straightforward way to do it, that would have a much higher ROI in terms of how long it would take you to implement. Yeah.

Noah Learner: Mary asked a question. I think she means. What are your? She said, Kristen, you are goals? And I think I think I think she means like, I don't think that's the question. I think she's saying that you're the bomb.

Kristin Tynski: I appreciate that. Thank you.

Noah Learner: Is that right, Mary? Is that? Is that what we mean? Looks like a yes. Chris. There you go.

Jordan Choo: Awesome. So Kristen, we're, we're coming to the top of our hour. And we do want to be respectful of your time. And thank you very much for for spending an hour with us. Oh, yeah. Anything else you want to add? before we say goodbye?

Kristin Tynski: No, you know, I enjoyed meeting you guys. And I appreciate the conversation. I'm glad you guys see, you know, the implications of AI NLP technologies in a similar way to how I see it. And, you know, I hope you guys continue that conversation. And I, you know, I appreciate you having me on and asking me about fractal too. So I had a lot of fun. Thank you.

Noah Learner: I think the we're going to have Jr, oaks on pretty soon. And you were, we were talking about Python dependencies. And he and I were chatting about something that he built. I was like, Hey, I can't figure out how to get this to work, blah, blah. And every time I asked him a question, I feel like a total dumb ass. And I said, You know, I feel dumb again. I said, Does this ever get easier? No. No.

Kristin Tynski: Totally empathize with that I'm here. I sit next to Matt Gillespie, who's our head of data science. And he's a very, very proficient Python programmer. And, you know, I'm always I'm constantly bugging him, as I've been learning. And you have, you have to kind of be a little bit humble and just be like, Listen, I know, this is a really, really dumb question. But, you know, it's gonna help me a lot here. So I probably just just take the approach of like, invest in trying to learn something that's fun for me, and then not be embarrassed about being an idiot and being okay asking for help from my colleagues that are much better at this than I am. Yeah.

Noah Learner: I'm pretty. I'm pretty blown away by this talk. And I feel like I couldn't focus enough and humbly apologize. Typically, I'm more on point because I don't have to. I'm physically holding the iPad steady the whole time, folks. Anyway, this was amazing. Kristen, you rock. Jordan. This was amazing. Thank you, everybody. We really appreciate you spending an hour with us. And we look forward to our next hangout in two weeks. Our guest is to be announced. We're very excited. We're talking to a couple people. Christian, have a great weekend. Jordan, you to everybody. Keep on automating

Kristin Tynski: You too guys.

Jordan Choo: Thanks, everyone. Thank you, Kristen.

Kristin Tynski: Bye