<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0"><channel><title><![CDATA[Engineering Management as it is]]></title><description><![CDATA[My personal reflections on Engineering, Management and Engineering Management and related topics]]></description><link>https://blog.filonov.dev</link><generator>RSS for Node</generator><lastBuildDate>Wed, 14 Oct 2026 19:21:09 GMT</lastBuildDate><atom:link href="https://blog.filonov.dev/rss.xml" rel="self" type="application/rss+xml"/><language><![CDATA[en]]></language><ttl>60</ttl><item><title><![CDATA[How You Benefit From Knowledge Sharing]]></title><description><![CDATA[This blog post was originally written and published on the inDrive blog. Want to work with us? Check our vacancies!

I'm addicted to learning and sharing knowledge. But it was not always like that. I always understood the profit of education, but for...]]></description><link>https://blog.filonov.dev/how-you-benefit-from-knowledge-sharing</link><guid isPermaLink="true">https://blog.filonov.dev/how-you-benefit-from-knowledge-sharing</guid><category><![CDATA[management]]></category><category><![CDATA[engineering-management]]></category><category><![CDATA[knowledge sharing]]></category><category><![CDATA[knowledge]]></category><dc:creator><![CDATA[Vladimir Filonov]]></dc:creator><pubDate>Wed, 01 Feb 2023 17:25:18 GMT</pubDate><enclosure url="https://cdn.hashnode.com/res/hashnode/image/upload/v1675271871676/343cce1c-2bac-4aa9-a916-be7ebc17e6c9.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<blockquote>
<p>This blog post was originally written and published on the inDrive blog. Want to work with us? Check <a target="_blank" href="https://indrive.com/en/career/">our vacancies</a>!</p>
</blockquote>
<p>I'm addicted to learning and sharing knowledge. But it was not always like that. I always understood the profit of education, but for a long time didn't get any reason to spend time on sharing.</p>
<p>Knowledge sharing has many forms: teaching, mentoring, blogging and attending meetups and conferences. Even an explanation to people of you your professional area your job. One of my colleagues worked in a Cloud Security firm. Once, his old mother asked:</p>
<p>‐ What is your current job, son?<br />‐ Well, we secure the Clouds.<br />‐ Why are they paying you for this?</p>
<p>As you can see, even words have different meanings for people. Sharing something you know can require extra effort, but it can also provide additional benefits for your own.</p>
<p>Today I want to share my reflections about what teaching gave to me personally (and I hope will provide to you).</p>
<p>I still remember the feelings and thoughts which stopped me from starting to share.</p>
<h3 id="heading-lack-of-purpose-and-motivation"><strong>Lack of Purpose and Motivation</strong></h3>
<p>This was the first and the biggest one. Why should I spend my time when I have so much to do and learn by myself. But in fact, this was caused by another thought.</p>
<h3 id="heading-what-can-i-teach-google-stackoverflow-knows-everything"><strong>What Can I Teach? Google / StackOverflow Knows Everything</strong></h3>
<p>The Internet is full of information in all possible forms. Free and paid, text and videos, superficial and deep. Do I really know something that is not there yet? And if I can find answers to my questions and solutions to my problems, everybody can, right? And again, it was not the root cause.</p>
<h3 id="heading-fear"><strong>Fear</strong></h3>
<p>It's always easy to sit quietly on the sidelines, so nobody will notice you. Because when we start to say something, there is always a chance that we are wrong, and somebody will start to argue. We are all scared of judgment. And in my experience, this is always the main obstacle - overcoming our fear. Later I realized that this was Trap #1.</p>
<h3 id="heading-the-trap-of-underestimation"><strong>The Trap of Underestimation</strong></h3>
<p>I will call traps states that are easy to fall into and very difficult to escape. So trap number one is an underestimation of one's own knowledge. "I don't know much; someone already teaches all this," and so on.</p>
<p>I began to overcome this trap (still not understanding what I was trapped in) by accident. I connected to several IRC (Internet Relay Chat) chats and the developer forum when I needed help. But gradually, I began to answer and help people myself in part I knew well. And then the discoveries began.</p>
<h3 id="heading-the-discovery-of-the-mess-in-my-head"><strong>The Discovery of the Mess in My Head</strong></h3>
<p>The first discovery was that the knowledge in the head was in absolute disarray. The paradox is that this mess is quite suitable for practical use. But to explain something coherently, you need to thoroughly dig.</p>
<p>The cause of that lies in the way we obtain our experience. Quite rarely do we have an academical learning process when things are learned from bottom to top step by step. In real life, we often compile solutions from pieces of knowledge and practical discoveries, and our mind stores them connected but with very blurry context.</p>
<p>But when I transfer this knowledge, I need to put it into logically sorted sentences, which requires clarifying the fog of the context I have in my head.</p>
<h3 id="heading-the-discovery-of-wordlessness"><strong>The Discovery of Wordlessness</strong></h3>
<p>Relatively fast, I understood that it's pretty hard to put things from my mind into understandable words. I can quickly provide the solution to the problem, but it's much trickier to explain all the details and get the right words to make it sensible.</p>
<h3 id="heading-the-discovery-of-obscure-the-obvious-is-not-obvious"><strong>The Discovery of Obscure. The Obvious is not Obvious</strong></h3>
<p>The most striking thing for me was that ideas and concepts that seemed evident to me, almost axiomatic and known to everyone, are by no means like that. But their obviousness gave me considerable gaps in explanations for others.</p>
<p>Our mind doesn't like excessive complexity and aims to simplify things. So, it removes the trail of how we learned something, primarily when it was obtained from practical experience. It's like it makes git squash from time to time.</p>
<h3 id="heading-the-benefit-of-classification-and-structuring"><strong>The Benefit of Classification and Structuring</strong></h3>
<p>All this led me to the need to work on myself. I had to systematize, classify and structure my knowledge. To do this, I made plans, wrote out definitions, and just tried to write intelligible textual descriptions.</p>
<p>After that, it turned out that more than simple systematization was needed. For each area, and each task, it was necessary to look for the correct and most effective form of submission. Moreover, it was necessary to remember about showiness because a clear but boring presentation needs to be corrected.</p>
<h3 id="heading-the-benefit-of-search-for-information"><strong>The Benefit of Search for Information</strong></h3>
<p>This gave rise to the following link - the need to expand my understanding of the topic, look for additional information, and read other people's articles and materials. In other words, to teach others, you have to study yourself. So some time passed, and I learned a lot and rejoiced at what a fine fellow I was.</p>
<h3 id="heading-the-trap-of-the-overestimation"><strong>The Trap of the Overestimation</strong></h3>
<p>And then I fell into a new trap. This is the feeling that you have finally gnawed Mjolnir out of the granite of science, that you have great power in your hands and in your head, and you need to spread it. And you start broadcasting. Not to explain, not to teach, but to broadcast one's knowledge like a radio, without any feedback.</p>
<h3 id="heading-the-discovery-of-a-feedback-gotta-learn-to-listen"><strong>The Discovery of a Feedback. Gotta Learn to Listen</strong></h3>
<p>Fortunately, I quickly noticed something was wrong, and knowledgeable people suggested that it was essential to collect feedback and, in general, build the process in the form of a dialogue, not a monologue. And it turned out that in addition to improving the methods, it also gave unexpected results.</p>
<h3 id="heading-the-benefit-of-unexpected-angles-in-looking-at-the-problem"><strong>The Benefit of Unexpected Angles in Looking at the Problem</strong></h3>
<p>The way how we see the world, including the problems that we face, is the superposition of our own experience. Often this leads to a particular route of looking at the issues and how we solve them. But most of the problems have multiple, almost infinite solutions. And since other people have different experiences, they can have a very different point of view and don't get why we choose ours. And there is nothing so refreshing as a new point of view.</p>
<p>Due to another experience, people start asking a very unexpected question. This brings us to the Discovery of obscure again - things we thought were obvious are not. A clear example of such a question for me was, "Why should I place a return in the function?" My first reaction was - "What do you mean, why? It's obvious!" But no, it's not. Can you easily explain why? =)</p>
<h3 id="heading-the-benefit-of-unexpected-difficulties-in-phrasing"><strong>The Benefit of Unexpected Difficulties in Phrasing</strong></h3>
<p>I will continue the example with the function. How many ways to explain function to the newbie do you know? At the very beginning, I had only a single form. But after explaining this in my first programming courses, I found out that many people need help to get this concept. More of that explanation that was fine to one person didn't work with another. And the opposite. Since then, I have been required to create about a dozen various ways to describe the same knowledge.</p>
<h3 id="heading-the-benefit-of-patience"><strong>The Benefit of Patience</strong></h3>
<p>Being calm when explaining something over and over again is a convenient skill, but it is tough to obtain. It requires a lot of practice but also brings many benefits since it overall affects your work and private life.</p>
<h3 id="heading-the-trap-of-routine-and-repeatability"><strong>The Trap of Routine and Repeatability</strong></h3>
<p>We love to make things that we are good at. We are starting to do them again and again. But unfortunately, satisfaction from doing them reduces, and one day we are hitting the point where we begin to feel boringness and tiredness. We want to do something new and refresh the process. But what else can we teach people if we already set up all our knowledge on educational reels?</p>
<h3 id="heading-welcome-to-the-spiral-of-cognition"><strong>Welcome to The Spiral of Cognition</strong></h3>
<p>At this point, you are underestimating your knowledge again, so you are trapped in the trap of underestimation. Way out of this trap is entirely the same - you need to start searching for the problems you know how to solve but have yet to try to teach somebody or explain the solution. This may be in the same area but at another level of complexity or in another field. Just keep seeking, keep sharing =)</p>
<p>By the way, is there something that stops you from writing a blog post? =)</p>
]]></content:encoded></item><item><title><![CDATA[Are you SHOKed?]]></title><description><![CDATA[Have you ever heard about a disease called Hydrocephalus?

Hydrocephalus is the buildup of fluid in the cavities (ventricles) deep within the brain
Source: Wikipedia

In medical history, there is one notable case: in July 2007 French 44 years old man...]]></description><link>https://blog.filonov.dev/are-you-shoked</link><guid isPermaLink="true">https://blog.filonov.dev/are-you-shoked</guid><category><![CDATA[johari window]]></category><category><![CDATA[KnowledgeManagement]]></category><category><![CDATA[knowledge]]></category><category><![CDATA[leadership]]></category><category><![CDATA[management]]></category><dc:creator><![CDATA[Vladimir Filonov]]></dc:creator><pubDate>Thu, 15 Dec 2022 12:23:12 GMT</pubDate><enclosure url="https://cdn.hashnode.com/res/hashnode/image/upload/v1671106367182/lhTaIAsn_L.jpeg" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>Have you ever heard about a disease called Hydrocephalus?</p>
<blockquote>
<p>Hydrocephalus is <strong>the buildup of fluid in the cavities (ventricles) deep within the brain</strong></p>
<p>Source: <a target="_blank" href="https://en.wikipedia.org/wiki/Hydrocephalus"><strong>Wikipedia</strong></a></p>
</blockquote>
<p>In medical history, there is one notable case: in July 2007 French 44 years old man was diagnosed with Hydrocephalus, with about 90% of his brain destroyed by liquid. And he was still able to live an almost healthy life.</p>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1671104552814/k0RA0pa77d.png" alt="Hydrocephalus. Copyright Feuillet et al./The Lancet" class="image--center mx-auto" /></p>
<p>It is incredible how brain cells slowly took responsibility for destructed parts and kept almost all functions! Of course, the process was prolonged and happened for 30 years, but still — an excellent example of sustainability. It became possible because of efficient information exchange between brain cells.</p>
<p>But let's try to apply this example to organizational structure. What will happen if one of your organization's "cells" disappears? If you lose all the information it contains will also disappear. Will any other "cell" will be able to take the first one functions? And if 10% of cells? And according to 10x thinking behavior, let's make a high bet - will your organization survive if 90% of cells are lost? Even if we will admit that we will replace them with new ones.</p>
<p>Each organization is a complex and multilevel organism — many people, teams, departments, and many dependencies. It is an excellent example of a live social network. And as any social network, it requires the flow of information to stay alive.</p>
<p>There are different levels of information - global, department-level, local, and even personal. I define Organization Knowledge as a summary of all this information spread across the company. And as with any flow, information spread can have bottlenecks and leakages, leading to various troubles.</p>
<p>For leaders, one critical task is to sustain the organization's structure. As engineers, we can compare it to a very complex mechanism. Let's take a rocket, for example. Rocket is working in a very unfriendly environment, and there are few possibilities to fix something on the run. That's why in such situations, engineers make duplicated schemas and nodes. If one will unexpectedly break - there is always a hot replacement.</p>
<p>Unfortunately, too often, I met situations when organizations had a lot of processes and knowledge in the hands of a single person or a small team. Get to know SHOK (Single head of knowledge) - one of the management anti-patterns. More broadly, it is known as Bus Factor.</p>
<p>So, how can we fight this dysfunction?</p>
<p>As always, we need to start with identification if the problem exists. I prefer the Johari Window model framework applied against all proper functions for all people in the company. Initially, this model was designed for better self-awareness and social representation, but it is also handy for knowledge spread analysis.</p>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1671105049790/wDGMTMvXR.png" alt class="image--center mx-auto" /></p>
<p>First, we need to identify the list of all critical functions in our business. It is quite a challenging but required task by itself. In fact, if you don't have such a list, you are already in trouble =) You will need to bring all your departments' heads together to build it as soon as possible.</p>
<p>Let's take the simplified example based on the Engineering department. Imagine you have agile methodology applied to the development lifecycle. We will skip planning and estimation of the tasks and will take only implementation and delivery:</p>
<ul>
<li><p>Task implementation (coding part)</p>
</li>
<li><p>Quality Control</p>
</li>
<li><p>User Acceptance testing</p>
</li>
<li><p>Backup</p>
</li>
<li><p>Deploy</p>
</li>
</ul>
<p>Once again - this is an oversimplified model, but I hope enough for an example.</p>
<p>So, we have the list. Now we can make a simple form to collect information across the company. There are many tools for collecting data, but for this example, I used Google Forms and Google Spreadsheet.</p>
<p>The form is relatively simple due to the oversimplification of our example. In real life, it will be much longer.</p>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1671105073807/eR79HIFmxm.png" alt class="image--center mx-auto" /></p>
<p>As you can see, there are two sections: "Can do" and "I know a guy." And no, "I know a guy… who knows a guy… who knows another guy" (c) does not work. Only direct relations =)</p>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1671105109711/3muaNXrd8.gif" alt class="image--center mx-auto" /></p>
<p>An outcome spreadsheet will look like this:</p>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1671105133179/SgFlB9QkL.png" alt class="image--center mx-auto" /></p>
<p>Honestly, I'm not a spreadsheet pro, so my methods are probably not brilliant. But they work for me =) BTW. I will be glad if you will send me a more elegant solution. So.</p>
<p>My goal is to create a Bubble Chart to visualize the high-level state of knowledge in the company.</p>
<p>On another sheet, I'm creating a new table with a list of all functions and columns representing Known/Unknown to self, Known/Unknown to others, and the bubble size.</p>
<div class="hn-table">
<table>
<thead>
<tr>
<td>Function</td><td>Self</td><td>Others</td><td>Size</td></tr>
</thead>
<tbody>
<tr>
<td>Task implementation (coding part)</td><td></td><td></td><td></td></tr>
<tr>
<td>Quality Control</td><td></td><td></td><td></td></tr>
<tr>
<td>User Acceptance testing</td><td></td><td></td><td></td></tr>
<tr>
<td>Backup</td><td></td><td></td><td></td></tr>
<tr>
<td>Deploy</td><td></td><td></td></tr>
</tbody>
</table>
</div><p>For self &amp; Others, I'm using such formulas:</p>
<pre><code class="lang-plaintext"># Self
=IF(COUNTIF('Form Responses 1'!$C:$C, "*" &amp; A2 &amp; "*"), 1.5, 0.5) + RAND()/3*IF(RAND()&gt;0.5, -1, 1)

#Others
=IF(COUNTIF('Form Responses 1'!$D:$D, "*" &amp; A2 &amp; "*"), 1.5, 0.5) + RAND()/3*IF(RAND()&gt;0.5, -1, 1)
</code></pre>
<p>Awful… But work in most cases =)</p>
<p>Since Google Forms stores multiple checkboxes answers to a single cell as a string separated with a comma, I'm just using a substring search to identify if any responder selected an option.</p>
<p>Why 1.5 and 0.5? I'm creating Bubble Chart with an x/y axis from 0 to 2, so 0.5 and 1.5 are just the centers of squares I receive. And to not let all bubbles overlay, I'm adding a random shift.</p>
<p>The formula to calculate bubble size is almost similar. The more people select this option - the bigger bubble we need.</p>
<pre><code class="lang-plaintext">=COUNTIF('Form Responses 1'!$C:$D, "*" &amp; A2 &amp; "*")
</code></pre>
<p>As an outcome, we will have a chart like this:</p>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1671105326820/cAhsMaeSq.png" alt class="image--center mx-auto" /></p>
<p><strong>The Upper left square is the "Open area."</strong> It means that at least some people marked both - they can do this function by themselves and know somebody who can do it also. Functions in the area are safe, and ideally, all bubbles should be here. The more people mention this function, the more giant bubble we have.</p>
<p><strong>The lower left square is the "Hidden area."</strong> Some people know how to do this, but nobody knows who else can. But even if the bubble is massive, there is a communication issue since nobody is sure that this function can be done by somebody else.</p>
<p><strong>Upper right - “Blindspot”</strong>. Everybody thinks that somebody else can do this, but nobody really mentioned that can — a hazardous situation.</p>
<p><strong>And the last one - "Unknown."</strong> Oops. Nobody can, and nobody can mention somebody. Have something in this area? Time to panic - something is going very wrong =)</p>
<p>Of course, this is not the ideal representation of reality, but it gives you an overall view of the situation. It can help you to identify weaknesses and knowledge loss in your processes. The more granular questionaries you will have, the better results you will obtain. This method works better if you collect function and data on a team level.</p>
<p>As I mentioned, identification of the problem is the very first step. And at this point, we have a general overview and can initiate the process of handling knowledge gaps and weaknesses. How to do this we will talk about in the following articles.</p>
<p>Start to collect data, and stay tuned!</p>
]]></content:encoded></item><item><title><![CDATA[Stop searching for goal-oriented candidates]]></title><description><![CDATA[I have acted as a hiring manager for various level positions for almost twenty years. Many different findings have been made, but today I want to focus on one.
Do you often see I’m goal-oriented/ We are searching for the goal/result-oriented in CVs a...]]></description><link>https://blog.filonov.dev/stop-searching-for-goal-oriented-candidates</link><guid isPermaLink="true">https://blog.filonov.dev/stop-searching-for-goal-oriented-candidates</guid><category><![CDATA[management]]></category><category><![CDATA[leadership]]></category><category><![CDATA[hiring]]></category><category><![CDATA[HR]]></category><dc:creator><![CDATA[Vladimir Filonov]]></dc:creator><pubDate>Fri, 21 Oct 2022 13:27:14 GMT</pubDate><enclosure url="https://cdn.hashnode.com/res/hashnode/image/upload/v1666358718834/XvUTodkTY.jpg" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>I have acted as a hiring manager for various level positions for almost twenty years. Many different findings have been made, but today I want to focus on one.</p>
<p>Do you often see <code>I’m goal-oriented</code>/ <code>We are searching for the goal/result-oriented</code> in CVs and job descriptions? I saw this quite often. More of that, for a long time I regularly said that about myself. But later, I discovered that this is more tricky than it seems.</p>
<p>Let's look deeper into people's brains and the psychological aspect of people's motivation. Wikipedia provides such a description:</p>
<blockquote>
<p><strong>Motivation</strong> is the reason for which humans and other animals initiate, continue, or terminate a behavior at a given time.<br />…<br /><strong><em>Motivation</em></strong> is derived from the word 'motive,' which denotes a person's needs, desires, wants, or urges.<br /><a target="_blank" href="https://en.wikipedia.org/wiki/Motivation">https://en.wikipedia.org/wiki/Motivation</a></p>
</blockquote>
<p>From the first view, it seems completely ok. But the devil is in the details.</p>
<h3 id="heading-goal-orientation">Goal orientation</h3>
<p>First of all, let’s assume - according to the term explanation, everybody is goal-oriented! And the real difference is made by the goal for which a person is oriented for.</p>
<h3 id="heading-persons-needs-desires-wants-or-urges">Person's needs, desires, wants, or urges</h3>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1666351600584/gidHExy8z.jpg" alt="Source: [Wikipedia](https://en.wikipedia.org/wiki/Maslow%27s_hierarchy_of_needs#/media/File:MaslowHierarchy.png)" />
Source: <a target="_blank" href="https://en.wikipedia.org/wiki/Maslow%27s_hierarchy_of_needs#/media/File:MaslowHierarchy.png">Wikipedia</a></p>
<p>Everybody is more or less familiar with <strong><a target="_blank" href="https://en.wikipedia.org/wiki/Maslow%27s_hierarchy_of_needs">Maslow's hierarchy of needs</a></strong>. It represents the hierarchy of personal needs, most often visualized as a pyramid with the most basic requirements at the bottom. Until lower levels are not covered, the higher levels will not be the case for an exact person.</p>
<p>In my experience, pyramid order can differ from person to person, but this applies only to high-level elements in most cases.</p>
<p>Let's see how this relates to goal orientation in practice.</p>
<h3 id="heading-i-want-vs-i-need">I Want vs I Need</h3>
<p>Imagine you are performing an interview, and the candidate says he/she/they is very interested in helping people in your product area. Sounds cool and very matching to what you are searching for, right? But after that, a person starts asking many questions about salary, insurance, paydays, etc. There is nothing wrong with these questions, but you need to be aware because if there are a lot of questions like this, a person does not feel safe.</p>
<p>He/she/they wants to belong (3d level in the pyramid) but needs safety (2nd one).</p>
<p>What does this mean in practice? In practice, this can mean that the main goal person will be oriented toward is keeping his\her\theirs position in the company beyond everything else. And keeping safe can lead to a lack of proactivity, standing up for his/her/their own opinion, and everything risky.</p>
<p>Once again - this is not bad by itself, and more of that - it can be useful for your company because such people can be very loyal (until events do not threaten their safety). But if they see (or even imagine) a possible threat, they will act to avoid it but not to meet company goals.</p>
<p>My main point here is that you should be aware of <strong>real goals</strong> of the person.</p>
<p>If the goal is to work less and get paid more - a person can do everything to be visible and verbal, but the outcome will be significantly low.</p>
<p>If the goal is to be pleased - a person can do overtime work and help everybody around in the prejudice of their tasks.</p>
<p>If the goal is to obey responsibility, a person can make everything to delay decisions, waiting for somebody to decide.</p>
<h3 id="heading-what-to-do">What to do?</h3>
<p>Unfortunately, there is no (or I don't know) stable framework for identifying people's subconscious needs during hiring interviews. But I can share some tips&amp;tricks from my experience.</p>
<p>First of all, you will need an <strong>Active Listening</strong> skillset. Long story short - this is a pack of techniques to improve the information you hear. People often say much more than they want and even more than they say. Nonverbal cues and emotional reactions - there is much information beyond the words.</p>
<p>Don't ask questions. <strong>Tell stories and ask for stories</strong>. Storytelling is an efficient technique to activate more brain parts. And it works in both ways - it also improves the involvement of the listener and teller. So, by telling a story, you will activate a person's brain, and by asking for a story, you will let him not answer bordered questions but include more personal interests and emotions.</p>
<p>Ask about a person's life and way into the profession - <strong>roots can say a lot</strong>.</p>
<h3 id="heading-conclusion">Conclusion</h3>
<p>Goal orientation is not so clear-cut since every action is goal-oriented. But the goal can be very different.</p>
<p>What you are really searching for is <strong>Business goals oriented</strong> person who has all basic needs covered (some of them by your company, by the way). But this is also not the best case. Business goals are quite often described as KPIs. And sometimes, meeting KPI does not lead to company profit and product value.</p>
<p>What is the best case? IMO - <strong>end-customer goals orientation</strong> is the best you can find. This means that person will think about solving real customers' issues.</p>
<p>But, as I already said, this is possible only when low-level needs are fully covered. And this is not a fixed state. The company should have a process and environment for permanent monitoring of the personal condition of employees for early identification of any changes in their needs.</p>
<p>If the company wants to have really motivated and end-customer-orientated employees, it should be end-customer-orientated by itself and, in the meantime - employees orientated.</p>
<p>Love your employees as well as your customers! ❤️ </p>
]]></content:encoded></item><item><title><![CDATA[A few words about memory profiling in Python]]></title><description><![CDATA[Memory issues in applications are quite common. Luckily, in Python you have to work with memory directly only when writing CPython extensions, so you have to deal with such issues less often. Using frameworks also reduces the risks of causing memory ...]]></description><link>https://blog.filonov.dev/a-few-words-about-memory-profiling-in-python</link><guid isPermaLink="true">https://blog.filonov.dev/a-few-words-about-memory-profiling-in-python</guid><category><![CDATA[Python]]></category><category><![CDATA[Django]]></category><category><![CDATA[debugging]]></category><dc:creator><![CDATA[Vladimir Filonov]]></dc:creator><pubDate>Mon, 04 Apr 2022 20:01:30 GMT</pubDate><enclosure url="https://cdn.hashnode.com/res/hashnode/image/unsplash/lYxQ5F9xBDM/upload/v1649102269497/vBs9w-ktl.jpeg" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>Memory issues in applications are quite common. Luckily, in Python you have to work with memory directly only when writing CPython extensions, so you have to deal with such issues less often. Using frameworks also reduces the risks of causing memory leaks.</p>
<p>However, understanding how memory is allocated in an application is always helpful. Let's see what possibilities we have with a small Django project as an example.</p>
<p>So we have a simple data model:</p>
<pre><code class="lang-python"><span class="hljs-class"><span class="hljs-keyword">class</span> <span class="hljs-title">Department</span>(<span class="hljs-params">models.Model</span>):</span>
    title = models.CharField(max_length=<span class="hljs-number">120</span>)

<span class="hljs-class"><span class="hljs-keyword">class</span> <span class="hljs-title">Course</span>(<span class="hljs-params">models.Model</span>):</span>
    title = models.CharField(max_length=<span class="hljs-number">120</span>)
    department = models.ForeignKey(Department, on_delete=models.CASCADE)
    description = models.TextField()

<span class="hljs-class"><span class="hljs-keyword">class</span> <span class="hljs-title">Lesson</span>(<span class="hljs-params">models.Model</span>):</span>
    title = models.CharField(max_length=<span class="hljs-number">120</span>)
    datetime = models.DateTimeField()

    course = models.ForeignKey(Course, on_delete=models.CASCADE)
</code></pre>
<p>And a dozen views that return JSON. Let’s review how memory will be consumed.</p>
<p>First of all, we need to see the overall picture. Let’s take the <code>memory-profiler</code> package.</p>
<p>It has a handy decorator that allows us to view memory dynamics in any function. Let's check one of the views:</p>
<pre><code class="lang-python"><span class="hljs-keyword">from</span> memory_profiler <span class="hljs-keyword">import</span> profile

<span class="hljs-meta">@profile</span>
<span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">get_lessons</span>(<span class="hljs-params">request</span>):</span>
   lessons = list(Lesson.objects.all())

   data = []
   <span class="hljs-keyword">for</span> lesson <span class="hljs-keyword">in</span> lessons:
       data.append({
           <span class="hljs-string">'title'</span>: lesson.title,
           <span class="hljs-string">'datetime'</span>: lesson.datetime.isoformat()
       })

   dump = json.dumps(data)
   <span class="hljs-keyword">return</span> HttpResponse(dump, content_type=<span class="hljs-string">"text/json"</span>)
</code></pre>
<p>By default, <code>@profile</code> outputs results directly into stdout. And it looks like this:</p>
<pre><code class="lang-shell">Line #    Mem usage    Increment   Line Contents
================================================
    29     59.7 MiB     59.7 MiB   @profile
    30                             def get_lessons(request):
    31     66.6 MiB      6.9 MiB       lessons = list(Lesson.objects.all())
    32                                 
    33     66.6 MiB      0.0 MiB       data = []
    34     69.1 MiB      0.3 MiB       for lesson in lessons:
    35     69.1 MiB      0.3 MiB           data.append({
    36     69.1 MiB      0.2 MiB               'title': lesson.title,
    37     69.1 MiB      0.3 MiB               'datetime': lesson.datetime.isoformat()
    38                                     })
    39                             
    40     70.6 MiB      1.5 MiB       dump = json.dumps(data)
    41     71.3 MiB      0.7 MiB       return HttpResponse(dump, content_type="text/json")
</code></pre>
<p>Absolute values are not as important as <code>Increment</code>. Alas, it is not always descriptive. For example, filling a list with data dictionaries in a loop will give an increase of 2.5Mb, although the sum of the increments shows only 1.1Mb. </p>
<p>If we want to check for a memory leak, data within a single call is not enough. We need to collect some statistics. And for this <code>memory-profiler</code> has executable <code>mprof</code>. It collects data and then generates charts like this:</p>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1649100532886/N-_wCine_.png" alt="image.png" /></p>
<p>Unfortunately, <code>memory-profiler</code> does not show deep details, so here another library comes to the help - <code>pympler</code>.</p>
<p>It allows us to get statistics about different data types. Let's look at an example:</p>
<pre><code class="lang-python"><span class="hljs-keyword">from</span> memtest.models <span class="hljs-keyword">import</span> Lesson
<span class="hljs-keyword">from</span> pympler <span class="hljs-keyword">import</span> tracker
tr = tracker.SummaryTracker()
lessons = list(Lesson.objects.all())
tr.print_diff()
</code></pre>
<p>Let's check what was created in memory when we received 10050 Lesson records from the database:</p>
<pre><code class="lang-shell">                                     types |   # objects |   total size
========================================== | =========== |
                               &lt;class 'str |       20172 |      1.30 MB
                              &lt;class 'dict |       20206 |      1.89 MB
                              &lt;class 'list |        9977 |      1.00 MB
                               &lt;class 'int |       21891 |    598.59 KB
  &lt;class 'django.db.models.base.ModelState |       10050 |    549.61 KB
             &lt;class 'memtest.models.Lesson |       10050 |    549.61 KB
                 &lt;class 'datetime.datetime |       10050 |    471.09 KB
                              &lt;class 'type |           9 |      9.68 KB
                              &lt;class 'code |          41 |      5.97 KB
                           &lt;class 'weakref |          25 |      1.95 KB
                       function (&lt;lambda&gt;) |           9 |      1.20 KB
                 &lt;class 'method_descriptor |          12 |    864     B
                 &lt;class 'getset_descriptor |          10 |    720     B
                         function (as_sql) |           5 |    680     B
                 &lt;class 'collections.deque |           1 |    632     B
</code></pre>
<p>We have a lot of strings. But it's okay - web protocols are mostly text.</p>
<p>But we have even more dictionaries. And their number is directly proportional to the number of records that we get — each instance has <code>__dict__</code> attribute. Also, we can see that for each Lesson instance there is also one <code>django.db.models.base.ModelState</code> (the class that stores the state of the instance) and <code>datetime.datetime</code> due to the presence of a <code>DateTimeField</code> in the model.</p>
<p>It is curious that if we get not only Lessons, but also related data using <code>select_related</code>, the picture of memory consumption will change:</p>
<pre><code class="lang-python">tr = tracker.SummaryTracker()
lessons = list(Lesson.objects.select_related(<span class="hljs-string">"course__department"</span>).all())
tr.print_diff()
</code></pre>
<pre><code class="lang-shell">
                                              types |   # objects |   total size
=================================================== | =========== |
                                       &lt;class 'dict |       70409 |     10.01 MB
                                        &lt;class 'str |       50323 |      7.87 MB
           &lt;class 'django.db.models.base.ModelState |       30150 |      1.61 MB
                                        &lt;class 'int |       51233 |      1.37 MB
                                       &lt;class 'list |        9980 |      1.00 MB
                  &lt;class 'memtest.models.Department |       10050 |    549.61 KB
                      &lt;class 'memtest.models.Lesson |       10050 |    549.61 KB
                      &lt;class 'memtest.models.Course |       10050 |    549.61 KB
                          &lt;class 'datetime.datetime |       10050 |    471.09 KB
                                       &lt;class 'type |           9 |     10.05 KB
                                       &lt;class 'code |          41 |      5.97 KB
  &lt;class 'django.utils.datastructures.ImmutableList |          28 |      2.22 KB
                                    &lt;class 'weakref |          19 |      1.48 KB
                          &lt;class 'method_descriptor |          12 |    864     B
                          &lt;class 'getset_descriptor |          10 |    720     B
</code></pre>
<p>The memory allocated for dictionaries has significantly increased. This is logical — after all, there are many more instances of various classes now. And for each instance, there are at least 2 dictionaries - for the instance itself and for <code>ModelState</code>.</p>
<p>You can find a lot of interesting and unexpected things in memory usage which will help to make your application more compact and effective. So I strongly recommend you to play with <code>memory-profiler</code> and <code>pympler</code> from time to time.</p>
<p>In the conclusion, here is a “small” image picture of the object graph of one Lesson instance, generated using the <code>objgraph</code> library =)</p>
<p><a target="_blank" href="https://cdn.hashnode.com/res/hashnode/image/upload/v1649100511658/Y1vuYJN8j.png"><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1649100511658/Y1vuYJN8j.png" alt="image.png" /></a></p>
]]></content:encoded></item></channel></rss>