Saturday, April 10, 2021

Lessons learned from e-EQE and looking to the future

Updated 18 Apr 2021: see also my post commenting on the epi discussion paper.
Updated 12 Apr 2021.
"It is only when the tide goes out that you see who's been swimming naked"
I already posted these comments on LinkedIn - this post gives a little more background. There was an enormous effort behind the EQE scenes, and a lot of people working many extra hours, but this could not prevent cracks appearing. 
On reflection, I think a few long-standing EQE issues may have played a role in creating the problems:
  • No effective communication with most candidates. Email is fine for sending standard messages, but there is little chance to request additional information. It is not always clear whether candidates actually receive them and understand the intention behind the emails.  A lot of candidates are surprised by changes in exam formats as not everyone prepares using a tutor, or with other students to discuss things.  
  • Secrecy and little transparency about every aspect of the exam and organization. Seen particularly by candidates who fail - it is very difficult to figure out where marks were missed, why the official answer was preferred, what alternatives were available etc. But this year, it was promised that the exams had been adapted to take into account the new format, but it was refused to provide any details as this was related to the content of the exams.
  • Just enough volunteers to make and mark the exams. No one expects perfection -  limited resources means that not everything can be addressed. But it is a professional exam, so it should be possible with sufficient preparation to have a reasonable expectation of passing. There is a minimum level that should always be achieved.
  • Few real limits for each paper in what can be asked. A real problem for many candidates is the habit of regularly introducing new things not announced before or tested before. It is unheard of in any other professional exam not to have an official detailed syllabus of what you need to know, both for the legal parts and the practical subjects. The REE is too vague to be useful, so candidates prepare based on unofficial study guides (like CEIPI or EQE Guide for Preparation) and doing old exams. There is still no clear distinction between legal subjects tested at Pre-Exam level and Main Exam level.
    And it is ridiculous that every year candidates need to have 2000 pages from the WIPO in case a question is asked about a PCT state (in many years, the WIPO and National Law are not needed at all).
  • Few real limits for each paper in length and style. Since 2017, an extra 30 mins was added to each Main Exam so that non-native speakers have more time. But there is no clear way in which this is enforced, and it is clear that the time has been eaten up by the exams. 
  • No effective way to assist those with En/Fr/Ge as a 2nd language. Non-native speakers have a clear disadvantage. The patent attorney terminology manuals are very useful, but reading/typing exams can still be slower. And giving everyone an extra 30 mins has now lost its effect.
Normally, these are compensated by a lot of flexibility in the marking, and that will be repeated this year. So, anyone well-prepared who generated a reasonable answer has a good chance (as in all other years). But moving forward, some suggestions of possible improvements:
  • Make e-EQE improvements highly visible, so that candidates know what to expect (platform + content) and can properly prepare. 
  • Using the new possibilities, better define the content to be tested and make sure it is relevant to daily work. 
  • Have a proper official syllabus for each exam - articles and rules, sections of Guidelines, case law, OJEPO's. 
  • Limit detailed PCT questions outside Europe to IP5 countries, and provide resources electronically.
  • Remove overlap in testing and preparation (candidates & committees). For example:
    • Pre-Exam: limit to only only legal questions, covering procedural and substantive law. Drop Claims Analysis (covered extensively in A, B, C). Use a mix of multiple choice and "D1"-style questions to test D1 level of knowledge. This is main test of legal knowledge.
    • Allow ME: A and ME: B to be taken anytime - either with Pre-Exam or Main Exam. These are practically oriented, relating to skills which are being developed early (drafting and amendment).
    • ME: A can be cut in half => reduce subject matter, focus mainly on independent claims,  provide less prior art.
    • ME: B is ridiculous with client giving you claims. Go back to old format, cut in half => reduced subject matter, only include a few claims, provide less prior art.
    • ME: D - limit to current D2 (D1 is now in Pre-Exam). But also include some basic trademarks/designs and basic trade secrets. Test legal concepts, and not substantive patent law (or greatly simplified substantive law).
      Expand basic foreign knowledge (JP, US) to include CN, KR (IP5). Also define clearly what you need to know. Possibly also test something about those foreign national patent systems. Knowledge of DE, FR and UK systems would also be useful.
    • ME: C has much repetition in attacks. Cut in half => fewer claims, provide a lot fewer documents. There should also be a proper infringement part in EQE - it makes most sense to add it here as technical understanding is important.
Perhaps also look even wider, putting the EQE in a broader perspective. A lot of effort is put into only passing candidates who are fit to practice, but there are many other people who interact with the EPO at different levels:
  • Permanent education required for qualified attorneys. For most, a lot of the knowledge is lost within a few months of taking their last exam. I know there were some attempts in the past to introduce this, and I know that no system is perfect, but something must be better than nothing. This would also address the issue of the varying knowledge levels of grandfathers \ grandmothers.
  • European formalities officer qualification. These are the people who have the most up-to-date procedural knowledge in the offices, but the levels of knowledge vary wildly. 
  • European patent agent qualification. Similar to US, where a basic knowledge of procedural and substantive law needed for those interacting with the patent office. This would be useful for people who do not wish to become patent attorneys (yet), or work in a company's patent department, or work in a lawyer's office.

22 comments:

  1. Dear Pete, this presupposes that the examination board has the capacity to be insightful. With today's behaviour, I unfortunately doubt this very much.

    ReplyDelete
    Replies
    1. I don't believe this. It is beneficial to all to have an efficient system that tests the right things in the right way.

      Delete
  2. The biggest issue for this year examination is providing a below-par system for EQE exams. It is clear that little adaption or thought went into making the papers to run smoothly online and it seems that the papers themselves were not sufficiently checked.

    They need to improve on quality and assurance. Candidates failing because they are confused about conflicting statements or unclear terms are not fair ways to assess candidates. Candidates failing because they struggle with claim formatting e.g. papers B and C are also not a fair way of assessment.

    ReplyDelete
    Replies
    1. I understand that there were technical limitations due to the platform chosen, so I can accept that there was no other way. But even under the old in-person system, or with a simple online system, the B and C would have been crazy, undoable exams within the time.
      I know that they sometimes deliberately include unclear parts to force analysis and discussion. I have no idea if this was deliberate this year.

      Delete
    2. biggest issue is that the unclarity this year was not just specified to one part. Paper C was so confusing without seeing the rest of the claims. It was the whole paper. The timing was also not right at all so I agree with you Pete.

      Paper A also had some confusing statements throughout the application.

      Paper B was the worse out of all of them for reasons that are very well documented already.

      I think the committee forgot how hard it is not to be able to have claims or the paper in front of them. I really do not see a reason why candidates cannot print the paper off next time. This surely must change.

      Delete
  3. I also found that the EQE committees are demanding more and more of our time with little justification or resemblances to the real examinations now. For example, there were 3 mock exams which eat into a lot of candidates time (away from family and work).

    What is disappointing is that the mock papers were substantially different (both in style and content) to the real exams. Take paper B - the mock exam had no resemblance to the real paper B. I thought the mocks were pointless in the end and didn't help much. We kept finding IT issue after IT issue and nothing appeared to be resolved e.g. no highlighting, no underlining, no formatting.

    ReplyDelete
    Replies
    1. I was also disappointed by Paper B, but if they can take it into account in the marking, it could be forgiven.
      I had also expected that Mock2 papers would be closer to the actual exams in style and content, but it seems as if they were made completely separately.

      Delete
  4. Having repeated some papers this year from 2019, I kick myself daily for not passing then. The eEQE 2021 was frankly unfair. I appreciate your sincere efforts in helping candidates and presenting here proposals for the future. But I am deeply dismayed by the lack of transparency and almost arrogance of the EQE committee throughout the last two years. Completely redesigning the eEQE for 2022 will be a nightmare for candidates over the next few years. I can only advise prospective candidates to stay clear of the EQE and the profession until the dust has settled.

    ReplyDelete
    Replies
    1. Coincidentally, I have heard from at least 2 people seriously considering a law degree instead of doing the EQE - afterwards you have more freedom to practice, and you are better prepared for litigation.

      Delete
  5. Fit to practice is the point of all of these exams, but I came away from the EQE feeling that there was no way that the EPO could judge me based upon these exams.

    I am left wondering why we are not doing infringement analysis and freedom to operate analysis for the client. This is a critical part of the work that I do. I would argue that a mistake here is going to cost the client as much, if not more, than not getting them broad enough protection.

    Pre-Exam forces candidates to take D right away. Then EPO views the Pre and an excuse to ask us any obscure question that comes to mind with the justification of "you had two years to learn this". The data from this year should give an idea of if the Pre actually makes a difference (I doubt it does).

    In the end, I feel that the current state of the exam is a type of arms race. The EPO is trying to write the exams in such a way that the methodologies of Delta Patents and other exam prep services do not provide much of an advantage.

    In terms of difficulty, I had expected an easier set of exams than normal in order to play it safe with all of the eEQE changes. This is not what happened.

    ReplyDelete
    Replies
    1. The argument that people prepare better over 2 years instead of 1 is not true for a lot of candidates. You forget a lot, and passing the PE can give the feeling that your legal knowledge is already enough.
      Sometimes methodologies expose weak points in exam structures, but mostly they are just showing a way to do a paper in the time available.
      I think the D exam has been relatively consistent over the years - a lot of the things they ask have been on before. Every year, they ask a couple of new things.
      I see more problems with the practical papers - there is no test of argumentation or judgement any more. It is just an enormous amount of reading and you have to just start typing.

      Delete
    2. As I see it, the purpose of the EQE is, above all, utilitarian for the EPO. It's not that much about being generally 'fit to practice'.

      In papers A and B, being combative and defending an objectively weak, or uncertain, position is always heavily penalized and likely leads to a fail.

      Paper A is about how to draft main claims that hopefully meet the requirements of the EPC from the get-go. This ignores the fact that clients may want to file a very broad application, possibly not new, and keep options open.

      To pass paper B, you mostly need to know A123(2) and A56 EPC, this is to ensure you understand a negative ESR or A94(3), give up and overcome the objections straight away without dragging it to oral proceedings..

      Delete
    3. About infringement analysis and freedom to operate analysis for the client, this is a matter of national law, out of the scope of REE, thus of EQE.

      Delete
    4. @Blink Patentability is also determined ultimately by national law. Following your logic, the EQE should be reduced to the same level as the US agent exam.

      I am looking at what you are expected to do in real-life. Obviously, if you want a complete answer, you need to consult other specialists and national experts. But you need to be able to form an opinion yourself, you need to be able to understand what they are doing, and you are supposed to be drafting infringeable claims when you write an application. I don't see the problem with requiring an infringement analysis.

      Delete
  6. I agree with your idea on D1 and Pre-exam. This is just busy work and is entirely redudant.

    My problem with C is not that we have so many documents (though one less is good), but that the EPO doesn't seem to understand the difference in time required between an inventive step attack that carried over from a previous novelty attack and one that requires an entirely new IS analysis. I know that they are trying to make the point that a dependent claim doesn't always have to same D1 as a sub-claim, but the amount of time that it requires to change to a new D1 + D2 combination needs to be respected.

    I find B to be the most difficult exam because the only thing that the client's suggested claim does is to make me waste time. I can't wrap my head around how the client is thinking, and that costs me a lot of time. Plus the client normally leads me in the wrong direction.

    With the A exam, I feel that the EPO has to play the game of "hide the essential feature". It's about interpreting the client's letter to figure out where they are mistaken or contradict their own letter. Because if the EPO is clear about what the essential features are, the novelty and inventive step analysis is normally pretty clear.

    The tests are not about "fit to practice", they seem to be about insuring that only a certain number of people are passing in a given year.

    ReplyDelete
    Replies
    1. The emphasis in C on closest prior art has gone of the rails - in real cases, you can just reformulate once the documents are admissible. They should also never have introduced partial problems because candidates now see that everywhere.
      I hate doing B - it is basically a "not fit for practice" exam => listen to the client and do what they say. And play it safe and go for limited claims. Perhaps they should also lok at the marking - in real-life, it is not fatal to have non-novel claims if your dependent claims are good.
      For A, they should perhaps also not deduct so many marks if your independent claims are not novel.

      It would also be better to move to a positive marking scheme for A and B - for C and D, you start with 0 marks and gather marks as you go. For A and B, you start with 100 marks and lose marks for every mistake that deviates from what they want. A and B tend to penalise things that can be fixed later (missing essential features etc).
      I have never seen evidence of a quota, but the exams are seen as a way of preventing less-knowledgeable candidates from entering. But as I mention, there are many other ways to have badly informed representation.

      Delete
  7. well at least the EQE committee has now confirmed that the present EQE is a joke!

    ReplyDelete
  8. EQE is one of the most elaborate and sophisticated professional exams out there. While admittedly the volume of information to sift through and the complexity of a paper can vary from year to year, this should only be expected from a non standardized exam. The EQE exams are as close to real life as they can be. The U.S. certainly does not come anywhere near close to assuring the same level of competence in the profession. Having a patent agent vs patent attorney practice silos sounds good but actually harms the integrity of the profession. But hey, the USPTO patent bar can be administered online every 30-days and after you pass you are no better off advising a client. If you have not seen evidence that the EQE ensures competence, I am sure you will see the evidence once the bar is lowered. Yes, non-native speakers (like me) are disadvantaged but so are people who cannot read or write or don't have a technical degree. The EQE is not about making sure people enter the profession as soon as possible. It is about making sure that those that do enter are professionals, and not just anyone who reads about parents for few months.

    Cutting some of the volume in B or C is probably very reasonable. As well as moving to a positive marking scheme. If you are going to test other IP rights, it should not be just basic things - don't see the point of asking if it is basic knowledge you can simply reproduce. Mandatory continued education can turn into a money making business for private companies. I think the market usually takes care of that as attorneys need to stay abreast of developments if they want to compete with their peers. But should such continued education be required, it should be for free and provided by the EPO. Finally, I support the idea of testing infringement and freedom to operate but that is beyond EPO's jurisdiction and the practices are inconsistent among member states. EQE is tough and I think it should remain tough and should continue to test application of legal knowledge and not just legal/procedural knowledge. It requires a lot of dedication and resources but it does test competency and it does prevent some not so knowledgeable to practice the profession.

    ReplyDelete
    Replies
    1. I fully agree that those who pass the EQE are knowledgeable and competent at that moment - I am just pointing out that it does not make sense to make the bar artificially high for that route, when other routes are not checked at all.
      It is a small percentage of "representatives". Approx. 700 pass each year, but there are 12400 currently on the "list" (EQE + grandparents). So, about 6% of the current profession have "peak competence".
      The list does not include:
      - trainees who are working on real cases anyway (working for a representative who signs or in some cases "signs")
      - lawyers (can work for any one)
      - employees (can only work for 1 client)
      - formalities officers (working for a representative who signs or in some cases "signs")

      The goal of education for a qualification is so that a client can be confident that someone with "European Patent Attorney" on the door. As with many specialist professions, the client has no way of judging your work or your knowledge - they can only rely on your qualification initially until they see how you handle cases.
      So, for the list, there are approx. 10000 who passed EQE more than 1 year ago. I estimate no more than 1 in 10 EQE passers put any effort in to keep up their legal knowledge to "EQE' standards after the exam => approx. 1700 coming even close to the knowledge level that was tested.
      Most EQE-passers build on what they knew and develop the more practical knowledge not covered by EQE, or become more specialised in certain parts of the procedure. This and experience is arguably more important for a successful case outcome than the EQE skills.

      The main point about having continuous education is to enforce the mindset that you should be continuously learning - passing an exam 5 years ago is not enough. The reason to have it mandatory is also practical - you need to reserve time (and some money) to learning each year. I think you also have the wrong picture about continuous education. It is not restricted to certain institutes - in the systems I have seen, anyone trying to get attorneys to attend can request recognition for a number of points. A lot of these are already free or relatively inexpensive (especially now after all the lockdowns - many free webinars with certificates and points). The EPO already offers a lot.

      Yes infringement can be complex: that is exactly why infringement should be taught and tested. This knowledge is much more useful than calculating the days left after interruption of proceedings or knowing the possibilities after successful entitlement proceedings ;-)
      Also, the exam is not an "EPO exam" - it is organized by the EPO/epi. The epi are jointly responsible for what is tested.

      Delete
  9. Thanks, Pete. I support continued education and maybe as a life-longer learner I tend to think that in a knowledge based profession like this people tend to educate themselves anyway but I am biased. Let's assume that half of the practitioners don't update their knowledge. I am not sure there is evidence that those people practice and harm clients. But let's assume that some of them do just that. I am not sure that few free webinars each year will prevent that harm. Maybe such webinars may mitigate some of it but is the benefit significant enough to impose such a retroactive burden on those that are already on the list? Would it be legal according to the EPC to impose this new requirement? I will have to do research. :-)

    The infringement/freedom to operate part should be taught and tested but the problem is how. Member states cannot agree on a European patent court. The desire for uniformity remains utopia - which infringement case law would be tested - German/French? Too much uncertainty for future candidates. Is a big revamp of the EQE necessary? I certainly agree that improvements to the EQE should be discussed and considered. However, I don't support proposals for ravamping the whole EQE just because there is some committee at epi that has been assigned the task to modernize it, now that we are digital.

    With respect to most of the particular improvements you propose above, I think they are very well thought out and should be seriously considered. Like having a proper syllabus and limiting somehow the PCT aspects or be more clear in scope. Also, very good suggestion is to limit the pre-exam only to D-style legal questions and have the pre-exam be only barrier to sitting paper D so that some people may choose to sit paper A/B along with the pre-exam. The pre-exam, at least when I took it, did not do a good job testing the claim analysis I think.

    Finally, it looks like the difficulty and obscure complexity of some parts of EQE only increase over time so this somehow should be moderated.

    Btw, the idea of patent agent qualification was introduced in the U.S. as a replacement of the apprenticeship requirement that still persists for EQE.

    ReplyDelete
    Replies
    1. I am the same - I am always learning anyway, even though it is not required. And most representatives will learn organically based on the cases that they have (you could give EQE-passers a 5-year exemption from permanent education).

      Yes, it does go wrong. I have seen a case where a representative (I don't know what kind - it was anonymized) requested a decision and filed an appeal instead of requesting further processing. (They lost the appeal). I have also seen older cases where strange things have happened.

      They have to change the EQE for the online format - shorter exam pieces are needed because of the health and safety rules. (the paper exams will never come back - it is too expensive).
      This means more modules, which also makes cheating less likely. So, there will need to be new exams anyway. Modules also give more flexibility to test in different ways and to spread the study load.
      So, decisions are now being made about what to test and how to test it - if that is only determined by what is easy to test and easy to mark, then everything will become multiple choice, and mainly procedural questions.

      I don't understand the resistance to testing infringement - inventive step is just as complicated.
      This is the main thing you get asked after qualifying. I am lucky that it was covered in the NL national exam (where they looked at DE, UK and JP rules), but not all states have national exams. It is usually pointless getting a patent that no-one can infringe.

      Delete