Friday, February 12, 2021

e-EQE 2021 update - 12 Feb 21 - Mock3 announced

© Matt
Update 16 Feb 21 - Mock 3 part updated. Last chance to test. At least test briefly on 23 Feb, even if you have not yet had any problems
Update 14 Feb 21 - AI monitoring part during exam amended. During exam, a lot is recorded and invigilators will have access. After exam, AI analysis will be done to flag problems, but people will review.
Update Feb 21 with a couple of things from Zsofia Pintz's LI post.
On 11 Feb 21,560 people followed an EPO/epi public webinar updating us on e-EQE. Panelists answered questions supplied in advance. There were a lot of unanswered questions in chat. Announced Mock3 for 23 Feb 21 as an extra test for your setup (see below). Pass on this info to all candidates you know.

Panelists: Xavier Seuba (EPO: Director Patent Academy and EQE),  Christoph Machwirth (EPO Head of EQE Secretariat), Simone Fausti (EPO Project Manager e-EQE), Tiem Reijns (epi Project Team e-EQE), Julia Gwilt (epi Chair of Professional Education Committee), Jens Horstmannshoff (EPO Project Team e-EQE). 

Thanks to the panelists for taking the time to do this. Hopefully, it will be online soon. I asked for the 28 Jan 21 session to be put online as well: these sessions really answer a lot of questions. Indented are my notes for things that seemed new or were emphasised, so check the official documents and my other posts.

General points

On 10 Feb 21, Ignacio Lobato Holtmann created a Telegram group for fast exchange of eEQE info. He also proposes forming using it study groups for the next EQEs. For a lot of people this is a more natural way of sharing info - they are answering each others questions and swapping tips. It is like a mini unofficial helpdesk. At the moment, there are 90 members, it is completely open, so members can also invite other candidates. So if you need help, join.
  • I am happy to have been selected again to do Paper D as an external bench marker. So I will also have to prepare my room, keyboard etc. and do the exam on 2 Mar 21 under the same exam conditions. 
  • 5 people are selected for each exam - the answers are used for discussions around marking. The answers do not determine marking levels in any way - they simply want answers scoring approx. 50 marks which they can discuss as a group about where to award marks or not. 
  • Read latest versions of official docs on official EPO e-EQE page and FAQ page in preparation for Mock3 and exam. Read in detail before testing and again before the exam (updated regularly)
    • a lot of questions are already answered here
    • they are working to combine everything in one guide (now available on e-EQE page)
  • 50% of candidates have still not tried the system. If you know somebody, please pass it on. Mock1 flows are available and there will be a Mock3 on 23 Feb 21. Mock 1 exams were made by the Examination Committees.
    • Real risk that you fail because something trivial like a camera does not work or your desk is facing the wrong way. It is your responsibility. 
    • If you "hand-in" an exam, flow is completed and you cannot start it again. So if you want to try again, don't hand-in for Mock1 or Mock3
  • Mock2 was intended to test everything with a big load, so flows are no longer available.
1. Location for exam
  • Make sure no windows, bookshelves, paintings, or light sources behind you.
    • Can confuse AI invigilation. If it is a problem, cover them with paper.
  • They can ask you to scan your room before you start with the camera to check everything
    • My suggestion: discussions on possible "suspicious behavior" may happen much later after the exam. I advise to video (or pictures) your room/desk/monitor before the exam, with date & gps location stamp. If something strange happens during the exam, type it in the chat. After the exam, if it was serious, let them know by email to helpdesk@eqe.org.
  • No glass walls or doors
    • Earlier, they said that you can cover glass walls and doors from the inside. Just sticking paper over them was suggested. You should close blinds and curtains. You make sure that no-one can see your screen or communicate with you.
    • Realise that for the whole exam, you have to sit at your desk in front of the camera. So if you have them covered and your desk is away from them, it should not be a problem.
  • No other electronic devices permitted unless explicitly allowed by Examination Secretariat in advance
  • You cannot have a 2nd computer or phone close by in case you have problems. 
    • Better to put outside the room
  • If you do have a second screen next to your first one but disconnected, you should move it way, well out of reach of your desk, and with the screen side not visible
  • Other computers may be in the room but must be switched off and out of reach
    • If the other computer has screens, disconnect them and move them away
  • Medical issues: you should have mentioned these issues and electronic apparatus when registering as they apply during a regular in-person exam. It is now too late to register and request special treatment
    • My experience: they consider reasonable requests, but no guarantees. Explain why you did not mention on registration. Don't request things you can live without. Be prepared to provide a letter from a doctor.
  • You cannot get up to get books from book shelves. You must remain sitting in front of camera for whole exam. Normal movement is fine, but head has to stay in camera range and with the camera capturing your ears as much as possible.
    • There was a discussion about whether books on the floor is allowed. Don't risk facial recognition issues - have all your books on your table or side tables if possible. Make sure you can grab them without moving your head out of range
  • Question asked about taking exam from US or though VPN. In principle, no restrictions, but you may get delays and connection may be less reliable. VPN should be avoided as it slows the connection down and might block Lockdown Browser.
CLICK BELOW FOR MORE (Invigilation section updated)

2. Hardware / software
  • Wireless keyboard and mouse
    • not yet in FAQs , but are allowed as an exception to the wireless devices ban (from a reliable source)
  • If you had technical issues during Mock2, check known Wiseflow technical issues, particularly troubleshooting and webcam troubleshooting
  • If you had issues with 2,3,4 finger gestures crashing Browser, on a touchpad, use an external mouse
  • Local copies: normally no local copy made on your system
    • You are given the option to save if the internet goes out
    Continue working if internet fails - avoid rebooting
    • this part seems to work pretty well in the Browser. It warns you and it will keep trying to reconnect. It will also give give you chance to save a local copy 
  • Hotspots - need to be set up so network is switched automatically. They have tried it with their phones. You will not be allowed to grab your phone to turn it on during the exam.
  • Someone had problems downloading the papers because the server addresses were blocked by their company network. Solution proposed was an unlimited network connection on day of the exam.
    • My suggestion: ask about a guest network. Most companies have them, and they are designed not to have access to the company internal network.
4. Invigilation
  • During exam, keystrokes, mouse movements, audio and images of your face are regularly sampled and stored. People will make the decisions about suspicious behaviour, not AI.
    • Avoid being flagged later by keeping face and ears within camera view and being quiet.
    • My research from other websites using Respondus: The image/audio etc data is stored, together with your interaction with exam (progress, start time, time handed in) and monitoring of internet and relevant hardware/software errors or reboots. 
      • This is used to create a profile, which is automatically analysed after the exam to determine the degree of suspicious behaviour. 
      • If you score highly, then people will review the incidents after the exam, look through all the data in detail, contact you and take any final decision.
    • Suggestion: if something happened, type in the invigilator window what happened to create a record. After the exam, if it was serious, send a report to the helpdesk@eqe.org explaining what happened.
  • During exam: if someone or something accidentally opens door, they will be lenient. Important is to immediately get rid of them.
    • Suggestion: type in the invigilator window what happened to create a record. After the exam, if it was serious, send a report to the helpdesk@eqe.org explaining what happened.
    • My research from other websites using Respondus: during exam, invigilators will have access to at least some of the data being stored. If they see something unusual, such as moving your head out of range a lot, turning your face away from camera a lot, or sounds, they can contact you. 
    • I have also seen reports that the Browser may automatically generate a warning if something is suspicious is detected, but it is unclear whether this will be used during eEQE. Don't assume that if you dont see a warning, you are okay. The data is analyzed in much more detail later. Avoid being flagged later by keeping face and ears within camera view and being quiet.
    • If they do not contact you during the exam, that does not mean that you will not be contacted later once the full data set is analyzed. Avoid being flagged later by keeping face and ears within camera view and being quiet.
  • No unscheduled breaks to leave the room during PE parts (4x 70 mins) and D1 parts (2x 90 mins). This to be sure that you have no contact with others to discuss the exam. If you leave room, you must first hand in your exam. Also cannot leave room or move out of camera view within first 30 mins of exam, even if you hand-in paper early.
    • My research from other websites using Respondus: by leaving the frame, your face is no longer recognised (or missing for so many frames) and this will be flagged to an invigilator. They may contact you.
    • I have also seen reports that the Browser may automatically generate a warning if something is suspicious is detected, but it is unclear whether this will be used during eEQE. Avoid being flagged later by keeping face and ears within camera view and being quiet.
  • During D2, A, B and C parts, you may leave the room to take short unscheduled breaks. Only possible after 30 mins passed from official exam start time. Inform invigilator via chat function:  “I take an unscheduled break” and “I’m back from the unscheduled break
    • My research from other websites using Respondus: by leaving the frame, your face is no longer recognised (or missing for so many frames) and this will be flagged to an invigilator. They may contact you. 
  • Also if you make noises, it is not a problem. But you cannot read out the questions aloud. "Thinking aloud" may be overlooked by the invigilator
    • I am not sure what thinking aloud is, but I guess it is muttering or whispering, with the occasional swear words (WTF) and sighs. Hopefully, this part can be tried during Mock3.
5. WISEflow and just before exam
  • Printing was too short. You will get 20min  now
    • Very welcome. Print from WISEflow from the regular browser before starting the Lockdown Browser. Make sure you have all the pages before starting Lockdown Browser.
  • You can already print the calendars for EQE2021 on EQE notices page
  • Mock1, Mock2, Mock3: For the flows, once you hand in a answer they are Closed and cannot be reopened. So if you want to play with the system again for that exam, you need to leave without handing in.
  • Make sure you hand in your answer during the real exam. There is no facility to do it in WISEFlow, so if you do not submit it, it will not be marked
    • There is a "Hand in blank" button in WISEflow  - don't use this. This hands in empty pages.
6. Lockdown Browser
  • See here for my unofficial guide to avoiding crashes. and ways of getting out of browser. Also explains one way that worked on my computer to change networks
    • Better to wait to see if original network is restored. You can carry on working.
  • Invigilator password: will be available only in real emergencies. It will not be provided beforehand
    • Needed to start late or exit browser after a crash or after an "illegal" act
    • Even if you have it, you are going to lose time, so make sure you take steps you can before exam to prevent issues
    • Explain why you need it
  • Alt-Tab twice kicks you out: they are looking at this. Try and stick to CTRL-C, CTRL-V and CTRL-SHIFT-V - these will be the safest
    • If tripped, you will need invigilator password, have to rest, costing you 5-10mins, and causing stress
    • Browser is a 3rd party product. Assume that this will not be improved and take action to avoid pressing. 
    • Like Matt (photo above), if you have a USB keyboard, you may be able to take of either the TAB or ALT cap. TAB is probably the one you don't really need. Laptop key caps cannot be removed!!!!!
      • Note that TAB / SHIFT-TAB can be used to change bullet indent positions. So you will only have one indent available. But that is enough fro the exam - dont try and make the formatting too nice. Use "-", "=", or space if you want to show a difference.
    • Or put a sticker on it so feels different. Or slide a thin piece of card, like a business card, under the edge. TAB seems to be the easiest to block as it is a wide key
    • William Salinger has invented a less destructive way using a plastic tab, particular for laptop keyboards.
    • I recommend also disabling WINKEY in the same way
  • Online reference docs: only EPO domain. No WIPO access. Still allowed to use any books you want.
  • EPO website has 2 versions of EPC Guidelines and EPO-PCT Guidelines. So when you search, you may end up in the draft version (which goes into force on 1 Mar 21). But that is not a problem. Answers based on the 1 Mar 21 Guidelines will also be accepted.
    • they are being very flexible on the versions of legal texts used this year
  • Copy/paste was not optimal in Mock2. The files were not produced like a real exam. Exam committee has taken more care, so it should be better
    • CTRL-SHIFT-V copies without formatting. Does not work on MAC - but there is an I icon on the right in the Browser toolbar which allows formatting to be removed after pasting.
  • There will be no way to hand in the exam from WISEflow. You have to do it in the Browser.
    • So if it crashes, you will need the invigilator password to start the exam "late" and start the Browser. 
    • In the tests that I ran, everything was retained in the answer, even after several reboots.
    • Also if it looks like things are missing, go to the REVISIONS overview in the Browser, showing the time and number of words.
6. Mock323 Feb 21 0930-1300 CET ME (B only), 1530-1640 CET - PE (one flow).
  • Mock3 is your last chance to test - this will be closest simulation of the real exam - if everything works, you can relax about the IT part 😊.
  • Purely for you to test your hardware and software setup - EPO will test entry pictures, Zendesk, invigilation. Not a full exam, and not a new exam. Purely for you to login and test. Please don't ask helpdesk about the invitation. You will get it.
    • Even if you had no problems before, the system may have changed slightly. You may you have new hardware/software as well. 
    • No need to do the exams - at least login on time, do the ID checks, start the Browser on time, try typing for 5-10 mins, move your head around like you normally do looking at your notes.
    • Try sending a message on Zendesk that you are taking an unscheduled break. Note that many do not have a chat window - test this in Mock3
    • I thought they mentioned that AI part of the invigilation will be tested, but I am not sure. Try moving as you would when you will be taking the exams. However, most of the analysis is done later - if your actions do not generate an error or an invigilator contacting you, that does not mean it will not be flagged later as possible suspicious behaviour (see above).
7. My suggestions: after finishing all your EQE exams
  • Put your TAB and WINKEY caps back :-)
  • Manually uninstall Lockdown Browser: in Win10, it is found in the start menu under Respondus
  • Be suspicious of any emails mentioning WISEflow or Lockdown Browser
  • If you get any emails about "updates" of the Lockdown Browser, do not click on them
    • the only official version is available after logging in to WISEflow
  • Preferably unplug all the extra cameras and microphones. Or at least cover the cameras when not in use. When LockDown Browser is opened, it continously captures images and sounds.
  • Take a couple of days off and celebrate as much as you can respecting your local lockdown rules ;-)
  • Don't read blog posts about model answers. They are usually made by EQE nerds who made the exams under relaxed conditions. They could still be wrong. Even if it is a perfect answer, no-one knows where all the marks will be awarded. 
  • The marking sheets are only finalised after the exam following discussions within the committees about what to accept and what not. For example, if there was an issue almost no-one dealt with, they usually compensate by awarding marks for something else that most people got. The exams also have built-in fail-safes, so if you miss something, you will not lose marks everywhere. 

113 comments:

  1. Thanks Pete!

    ReplyDelete
  2. Great! Thanks for the Mock3! I actually thought that this must be coming! However, surprise that only half have test the system!

    Now I can test whether CTRL-SHIFT-V works also in Firefox!

    ReplyDelete
  3. Great tips Pete!

    'Don't read blog posts about model answers. They are usually made by EQE nerds who made the exams under relaxed conditions. They could still be wrong. Even if it is a perfect answer, no-one knows where all the marks will be awarded.'

    This made me laugh, but I really agree. Once you hand-in you can't change anything, why torture yourself? Life's too short. You pass (hurray) or you fail and get to have another go in 2022.

    ReplyDelete
    Replies
    1. As an EQE-nerd, I can say that :-)
      For my EQE 2006, I missed a first application problem in D2, I screwed up a priority question in D1 and still passed. But many other candidates missed it, so they shifted the marks somewhere else.
      I did not attack on independent claim on C and still got a good score because I did the argumentation for the other ones in detail.

      Delete
    2. Yes, good anecdotes.

      I intend to download my paper from each (in case they are required for whatever reason: force majeure!), but I will do as I did for the UK exams. Put them in a folder and forget about them unless they're needed!

      Delete
  4. I was unable to attend the webinar this week as it was full but I am slightly concerned regarding the bookshelves portion. Are they saying that bookshelves must be entirely removed as this is not exactly trivial! I am trying to remain as understanding of their position as I can, but they seem to be requiring a large amount from people who don't have the benefit of having a dedicated home office and seem to assume that all trainees will have access to this

    ReplyDelete
    Replies
    1. No - only if they are behind you, it can confuse the AI. But you can just cover them up with a sheet or paper.
      The other point is you cannot get up and get books during the exam from a bookshelf out of range.

      Delete
  5. It's really welcome that full invigilation will be tested - it seemed that the AI invigilation was not (fully?) switched on for Mock 2 of the pre-Exam. I really want to be able to test if the background noise from builders in my neighbour's house will cause any issues.

    ReplyDelete
    Replies
    1. I have updated the post regarding invigilation. Likely that all the data is recorded. It is unclear the system will generate warnings at all. Also unclear whether you will get feedback later on whether something was suspicious.

      Delete
  6. EPO has just uploaded a consolidated guide and a step-by-step Guide. I have not checked to see what is new.

    ReplyDelete
    Replies
    1. Thank you very much for your efforts to help us during this EQE season. It is obvious that you are helping us to make the best out of this situation.

      Delete
  7. Is the 20 mins printing for all main EQEs exams?

    ReplyDelete
    Replies
    1. Yes - thyy have included it in the new step-by-step Guide. A big improvement!

      Delete
  8. I don't have a big desk so I will need to put some books on the floor. As long as it only takes 2 seconds to pick up a book, I really don't see why this is a problem.

    ReplyDelete
  9. There a big folders that are needed for paper D so it should be OK to have it on the floor in case you need it. Maybe they can reduce what we need to bring for paper D like a Nat law table which has over 1000 pages. PCT applicant guide and national phase entry states are also quite bulky.

    ReplyDelete
    Replies
    1. The Respondus system uses AI - so the AI decides what is acceptable or not.
      The EPO seems to have little control over this. They can control the response by the invigilators, who will be lenient.
      But it seems likely (but we don't know for certain) the AI will give you a warning, and at some point decide it is too much and lock the browser. You will then need the invigilator password to continue, losing at least 5 minutes each time.

      Delete
    2. The AI inviligation is so inflexible. Why can't we also trust EPA attorneys to inviligate candidates. Do we no longer trust European patent attorneys. They can sign a statement to say theg will be impartial or use an attorney who you don't know.

      Delete
    3. After discussion with others who followed the webinar, Sorry - I misunderstood how sever the AI will be monitored. I have updated the invigilation section in the post above.

      Delete
    4. Sorry for the confusion - I misunderstood how severe the AI will be monitored. I have updated the invigilation section in the post above.
      Basically, the AI will register everything, and the invigilators will have access to part of the data.
      I do not know if there will be automatic warnings, but you should not assume that if you do something during Mock 3 or during the exam and there were no issues, that it was accepted. The main criteria seem to be: keep your face and ears within camera range as much as possible, and keep the room as quiet as possible.

      Delete
  10. Is there a reason why you can't read out the question? I often do this.

    ReplyDelete
    Replies
    1. They are afraid you are communicating the question to someone else or talking to someone. Also, the AI monitors the voice, so if there is too much noise, it will kick you out. They said that an occasional noise will not be a problem.
      Talking out loud would also not be allowed in an in-person exam.

      Delete
    2. For goodness sake. I am quite frustrated about all of this. I wouldn't say I shout out the question but I do read it out to myself. Sufficient so that others around me won't hear me mutter the question to myself in an exam hall. my speaker can detect very small sounds/noises so I may have to change its sensitivity.

      Thanks Pete. It is nothing against you.

      Delete
    3. I understand the frustration - unfortunately, the Browser suppliers are very secretive about what they do and how. They should really have a test site where you can try out your system.
      They treat everybody as if they are trying to cheat, but most people just want to make sure that everything will work on exam day.

      Delete
    4. I updated a couple of things based on Zsofia Pintz's LI post. They said thinking aloud may be overlooked by the invigilator, but reading the questions and/or the text aloud is not allowed. I think if you mutter occasionally, it should be okay. I hope it is tested in Mock3.

      Delete
  11. Pete, you are saying: your desk is facing the wrong way. It is your responsibility. - I take this example.
    Ok, it's mine. I did 4 main Mock1 at exam conditions, then last week I sat all 4 Mock 2-did 5 photos: one before exams and 4 on every Mock 2; I wrote an e-mail to helpdesk and I still do not know whether my desk is facing the wrong way as I do not have an answer from helpdesk and did not receive any sign throught Mocks2.
    My desk is infront of cupboard with a lot of books, where is no face/picture and should not be a problem but I'm not sure on real exam if AI will do a problem after all Mocks and the 3 meetings with EPO I participated. Do you see my point? I did all due care but still am not confident that exam will be ok.
    The desk is just an example, there are a lot of such tiny little issues which can go wrong on real exam after all the time and enery spent from my side.
    Leaving aside 1 year of studying for all main exams

    ReplyDelete
    Replies
    1. Hi Mariya, the Ai recognition is frustrating we have not really been able to test it. Try it on Mock3 and it should let you know if there will be problems. It depends how close it is - there needs to be enough contrast between your head and the background.
      But you can just hang up a sheet over it or some paper. It is an open book exam, so the presence of books is not a problem.

      Delete
  12. I was under the impression that AI does not decide what to do and only a human inviligator can decide. So with that logic, how can the AI lock you out of a system. Surely it is better that AI flags to the human inviligator that you are outside the camera view and let the human inviligator decide to lock out the candidate. This should not be an instant automatic lock out of the system by AI if you bend down to pick up your pen for example.

    ReplyDelete
    Replies
    1. I don't know how sensitive it will be, but the AI will decide to warn you and then kick you out. Exactly the same as with Alt-Tab - no humans are involved.
      They have chosen a highly automated system because they will have 2000 candidates making the papers at the same time.
      So the invigilator will decide whether to give you the password or not for the recovery.
      After the exam, they will review the incidents, and people will decide, not AI, whether the actions are suspicious. They sample the camera and the microphone regularly to they have a lot of data. They also have a record from your earlier tests of the system.

      But they are going to be reasonable, so they will give you the benefit of the doubt. So if you trip these things a few times, it will not be a problem. It will just cost you a lot of time.

      Delete
    2. In my opinion, automatic decisions by the AI that throw you out of the exam without an intervention of an inviligator will be catastrophic.

      Think about the unscheduled breaks - I highly doubt that there is any connection between the AI system and the Zendesk (and it can also not been switched of manually, because they state you do not have wait for a response of the inviligator for taking such a break). For the AI it will look like that you are several minutes away from the screen, which seems to be a very hard incidence (see above, candidates worry about getting some literature from the floor being out of the camera for just seconds, and EPA recommends not to view to far away from your screen).

      What happens if someone is kicked out of the browser because of this? Even if you get the inviligator password (might take some time if there are several people having problems), you might need to reboot your system or at least the Lockdown Browser.

      In my opinion, it is grossly negligent to not test the AI inviligation before. We saw what happend with the ALT-TAB problem (nobody was aware of this before Mock 2).

      Delete
    3. @Benedikt Good point about the unscheduled breaks - candidates should try it it during Mock3.
      Yes - with maximum leniency, you will get the password through the chat. But you will still be rebooting regularly.
      They were testing this on Mock2 - but they also needed the maximum amount of people to try it. They did not realise the sensitivity - that was "decided" by the 3rd-party software. That is why they did not have enough invigilators.
      I believe that they tested the AI internally as they used that to set up the guidelines. But only with a few people.

      Delete
    4. Sorry for the confusion - I misunderstood how severe the AI will be monitored. I have updated the invigilation section in the post above.
      Basically, the AI will register everything, and the invigilators will have access to part of the data.
      I do not know if there will be automatic warnings, but you should not assume that if you do something during Mock 3 or during the exam and there were no issues, that it was accepted. The main criteria seem to be: keep your face and ears within camera range as much as possible, and keep the room as quiet as possible.

      Delete
  13. Sounds like you are not even allowed to cough during your exam - I think the use of this software has been a bad approach by the EPO in my opinion. We are like guinea pigs for an experiment this year in preparation for candidates next year. There will be a lot of appeals no doubt.

    ReplyDelete
    Replies
    1. Also realise that without this, the EQE would have already been cancelled. The pandemic restriction are even worse than last year.

      An appeal will only help you personally if you have a borderline number of marks.
      They are not going to award a pass based on the Browser crashing and not handing in an answer or a partial answer.
      You could get your money back, or the EPO could be forced to improve, or forced to rerun the exam. An appeal will take at least 6 months to decide.

      So, generate the fullest, most correct answer that you can, and then consider appealing if you miss a few points, or you have a solution they did not accept.

      Delete
  14. Has anyone tried staring away from the computer for a while and then looking at the camera again. You are still in shot but if you want to take a mini break from the screen and turn your head away, would the AI lock you out?

    I agree with others here that this AI software automation system is nonsense and is creating more problems for candidates.

    ReplyDelete
    Replies
    1. Apparently that part was only turned on for a few people during Mock2. Not for me. It would be great if someone could post about their experiences.
      But you are not required to stare at the screen or camera for the whole exam. You have to sit in front with your head in view (particularly your ears), but you can look down and side-to-side.

      Delete
    2. I tried unsheduled breaks during A, B, C during Mock 2 - so away from computer but did not receive any sign from AI or invigilator - I don't know if AI was working

      Delete
    3. Try it during Mock3 - that will be the closest to the real exam.

      Delete
    4. Sorry for the confusion - I misunderstood how severe the AI will be monitored. I have updated the invigilation section in the post above.
      Basically, the AI will register everything, and the invigilators will have access to part of the data.
      I do not know if there will be automatic warnings, but you should not assume that if you do something during Mock 3 or during the exam and there were no issues, that it was accepted. The main criteria seem to be: keep your face and ears within camera range as much as possible, and keep the room as quiet as possible.
      Just because something is not flagged within Mock3 does not mean that it will be allowed.

      Delete
  15. As always, thanks a million, Pete! During the Mock 2 D, I had all of my D stuff on the table, and it was really easy to never leave the webcam area.

    the only crashes I have had are Mock 1: I work for a while, take a break, come back, and it tells me I've crashed it. So, never quit typing ha ha.

    Another lesson - I was wearing my fitbit. Didn't even occur to me until I was in the middle of Paper D - that thing can receive iMessages! So, WATCH OUT, Fitbit wearers!!

    ReplyDelete
  16. I see a lot of comments on the AI system. As far as I understand, the AI only signals to the human invigilator that something may be off (e.g. no face anymore in the field of view of the camera). It would of course be impossible to have human invigilators look at all camera feeds at all times.

    If you want to take a break, you just tell the invigilator by chat that you are leaving the room. When the invigilator then receives a notification from the "AI" that there is no face in the field of view of the camera, the invigilator can read the chat and understands that you have left the room. When you re-enter the field of view of the camera, one may expect that the invigilator will ask you to do a room sweep.

    As to a lot of other questions, which were also mentioned during the last webinar of the EPO, I think applying common sense solves a lot! No, of course you cannot wear headphones which allow communication with others, and of course you cannot read aloud the questions because this would allow a third party to answer the questions for you. Would you have worn headphones or read aloud the questions in the old analogue situation? No.

    ReplyDelete
    Replies
    1. Jan speaks sense!

      Delete
    2. @Jan I hope it works like that in practice, but the info is bit contradictory. They were saying on the webinar that if you stood up, you would be thrown out. And that if you were getting things from the floor, bending down, you could also be thrown out.

      So there are two types of invigilator - AI-triggered ones (who respond quickly) and Zendesk triggered ones (they said can take 5 minutes if it is busy)?

      I hope we see it during Mock3 - then it will be clearer.

      Delete
    3. I recall from the webinars that it was stressed that no decisions will be taken by the "AI", but instead decisions were always made by a human! The "AI" is only used to give an alert to the human, and it is up to the human invigilator to act. The alert may be based on your camera feed or sounds detected by the microphone.

      As such, I do not expect any throwing out of candidates who are out of view for a couple seconds, or who announce that they are taking an unscheduled break.

      See also point 5 of this document on taking unscheduled breaks:

      http://documents.epo.org/projects/babylon/eponot.nsf/0/3E0DC8281DE60DCDC125867A0041E3DC/$FILE/eeqe_candidate_information_compilation.pdf

      "You must inform the invigilator via the chat function when you take an unscheduled breakand when you return to your desk e.g. “I take an unscheduled break” and “I’m back from theunscheduled break”. You do not have to wait for authorisation from the invigilator beforeleaving your desk. Make sure you leave the system/browser running"

      Delete
    4. Further to my comment above, I believe that unscheduled breaks are not allowed during the pre-exam and the two D1 parts (source: http://documents.epo.org/projects/babylon/eponot.nsf/0/66EB601464EC7BECC12586320050164F/$FILE/ExamPapers%20EQE%202021_EN.pdf)

      Delete
    5. I just want to add that there are also many many issues with the wiseflow itself. Alt tab and other common keyboards commands don't make sense. No highlighting in Pdf which is standard. Copy and paste is horrible. Table of content doesn't work in wiseflow. Issues with printing time and making papers available on time. These standard things needed to be sorted before they are rolled out to candidates but they weren't. So I can completely understand why confidence is so slow. They have done a bad job of this by making it far too complicated.

      Delete
    6. The issue is that AI are throwing out candidates e. g. By using alt tab command. I was also under the impression that a human will decide but during mock 2 this was clearly not the case.

      Delete
    7. I don't think that the alt-tab issue and the AI are related; as far as I understand, the AI monitors the camera feed and microphone. I have not heard anything on the AI monitoring keystrokes. That said, the fact that the alt-tab issue even exists is quite sad.

      Disclaimer: I am a candidate doing the main exams, trying to get as much information as possible. I really appreciate the effort that Pete puts into this blog!!

      The blogposts and especially the comments are in my humble view to slanted towards the negative aspects. I appreciate that this may be different for others, but I for one am really happy that I can use my computer for the exams. For example:
      - I don't have to worry about my poor handwriting, which will inevitably happen after hours and days of writing;
      - ctrl+c, ctrl+v instead of scissors and glue;
      - the true magic of the ctrl+f function in the exam, this helps me a lot!! Especially in the C exam this is absolute gold, to search for definitions, which annex discloses a particular effect or feature, etc. Also in D2, this allows may to quickly check where certain subject-matter is disclosed in the client's letter.
      - Being able to search the guidelines, case law, OJ posts, Natlaw tables online saves me time and allows me to find what I'm looking for faster.






      Delete
    8. @Jan. Keystrokes and mouse movements are monitored, but the ALT-TAB is more "A" than "I" ;-)
      Good to have positive comments and that is what you want to do. Use the advantages of the system.
      Just being able to type is a huge advantage for a lot of people.
      You can also copy paste parts of your answer, which can be quicker where you have repetitive structures (like inventive step etc). And search.
      These people probably have more chance of passing this year than with the old system.

      Delete
    9. I agree that it's good to go online (finally) as other professions have done so many years ago.

      However, what has been dissapointing is the hurdles of the online system the supervisory board insisted on. Does it really need to be this complex. You can print this but not that. You must use wiseflow which is fine but nothing useful is available like highlighting function.

      Most candidates want online and welcome online exam but it's fair to say that none of us expected it to be this complex and difficult. Updates and changes keep happening and guideline changes very late till the exam. These things take so much time that its eating into alot of candidates preparation time.

      I would hope mock 1 and 2 would solve some issues and ease my concerns but it has only highlighted many issues. Let's hope they finally cracked it with mock 3. But its clear that more changes are required after EQE 2021.

      Delete
    10. Sorry for the confusion - I misunderstood how severe the AI will be monitored. I have updated the invigilation section in the post above.
      Basically, the AI will register everything, and the invigilators will have access to part of the data.
      I do not know if there will be automatic warnings, but you should not assume that if you do something during Mock 3 or during the exam and there were no issues, that it was accepted. The main criteria seem to be: keep your face and ears within camera range as much as possible, and keep the room as quiet as possible.
      Just because something is not flagged within Mock3 does not mean that it will be allowed.

      Delete
    11. I only study now with Mock 1. I figured out how I crash it, so I won't ;) As for the 50% who haven't logged in, maybe they can boost our pass rate?

      Delete
  17. Thanks for the post Pete! However, I still have a question regarding for which you may now the answer. Because of the CoVid pandemic, some of the OJs defined general dislocation periods. In the exam: should we apply directly the extensions defined by the OJs even though the statement of the question doesn't refer to the pandemic? For the main examination, it would make sense because you can provide reasoning for your answer, but for the pre-EQE? What do you think?

    Thank you

    ReplyDelete
    Replies
    1. You are welcome!
      Just use the calendars they give you for the main closing days.
      If they have a question on such a closure, they will give you all facts in the question, so you don't need to find the OJ EPO notice.

      Delete
  18. I'm still slightly confused about what to do with notice of opposition. Do we just write it out twice, one for part 1 and one for part 2.

    ReplyDelete
    Replies
    1. Together, part 1 and part 2 need to include the details required for an admissible notice of opposition. "Notice" is not the same as the "form" - you do not need everything that could be mentioned in the form.

      Delete
  19. Are we allow to eat and drink or is that too loud for AI to cope with and thus, throw us out of the exam?

    ReplyDelete
    Replies
    1. The Guide says that food and drink are allowed.
      Try eating Doritos and drinking a milkshake during Mock3 to see what happens.

      Delete
    2. Sorry for the confusion - I misunderstood how severe the AI will be monitored. I have updated the invigilation section in the post above.
      Basically, the AI will register everything, and the invigilators will have access to part of the data.
      I do not know if there will be automatic warnings, but you should not assume that if you do something during Mock 3 or during the exam and there were no issues, that it was accepted. The main criteria seem to be: keep your face and ears within camera range as much as possible, and keep the room as quiet as possible.
      Just because something is not flagged within Mock3 does not mean that it will be allowed.

      Delete
  20. After discussion with others who followed the webinar, Sorry for the confusion - I misunderstood how severe the AI will be monitored. I have updated the invigilation section in the post above.
    Basically, the AI will register everything, and the invigilators will have access to part of the data.
    I do not know if there will be automatic warnings, but you should not assume that if you do something during Mock 3 or during the exam and there were no issues, that it was accepted. The main criteria seem to be: keep your face and ears within camera range as much as possible, and keep the room as quiet as possible.

    ReplyDelete
    Replies
    1. Regarding doing things during Mock 3 - I spoke to my partner, played on my phone and didn't face the camera for large parts of mock 2. So I just assume the AI wasn't even switched on.

      Delete
  21. Hi Pete, I have a book shelf behind me but had not read anything about that being an issue. It is pretty big and hard to cover - there are no faces on it, can I ask whether this could be problematic and how you heard this? Thanks :)

    ReplyDelete
    Replies
    1. It depends on how close it is and the contrast with your face. I am not sure if it will be tested in Mock3 - it is possible they monitor the facial recognition, but I dont know if you will get any feedback. But during the exam, it is the main thing they monitor, as well as sound.
      Can you cover at least a part with a sheet or paper?

      Delete
    2. Thanks, I have ordered some Bluetac and A3 paper - it will take a while but I will do that just to be safe! Thanks!

      Delete
  22. I am still confused how to indicate deletions in the B Paper.

    Should we delete i) [text to be deleted] or ii) only [].

    While i) indicates the text to be deleted in the amended claims, ii) does not indicate the deleted part.

    What would be the best strategy here?

    ReplyDelete
    Replies
    1. Whatever is easiest for you to keep track. The markers just compare your claim set with their answer. I prefer to see what I have deleted while I am arguing the support.

      Delete
    2. thanks for your opinion Pete!

      Delete
  23. I was just informed by the Helpdesk that it is not allowed to use a wireless keyboard and mouse!

    ReplyDelete
    Replies
    1. I sure hope that such is not the case. Stay tuned for the next FAQ, I suppose :(

      I could run out and buy new equipment if I had to, but I would prefer to live with the rig that I have been practicing on for the last month.

      Delete
    2. You are the first to mention it.
      I have a wireless mouse, like 90% of candidates and 100% of offices. I did not have any problems at all during Mock2 (but I do have a wired keyboard).
      This is ridiculous.

      Delete
    3. I don't understand how this can be the case. Surely they will understand that a lot of people have this set up to prevent too many cables/not enough USB portals. Can you please post your response in the forum? I have sent an email about this as to reset up my home set up would be a nightmare!

      Delete
    4. Most new laptops have just a few USB ports. I heard from someone who had to take their laptop out of the docking station because they were getting errors in the Browser. Correct - this will be a real nightmare.

      Delete
    5. I have double checked the FAQ's and the requirements (http://documents.epo.org/projects/babylon/eponot.nsf/0/4C1F8318D652D9BBC125866B003B6D51/$FILE/Requirements_eEQE_EN.pdf) and they mention "Apart from the computer system (screen, keyboard, mouse, etc.), router and printer needed
      to take the examination, no other electronic devices (e.g. calculators, digital watches,
      smartphones and smartwatches) are permitted unless explicitly allowed by the Examination
      Secretariat in advance." There is nowhere in this document that I can see that states that wireless equipment is not provided. I was unable to go to the webinars, did they mention this on there?

      Delete
    6. I have also done mock 1 and mock 2 with wireless keyboard and mouse and had no problems with that.

      Regarding my inial post, I have specifically asked:

      "Is it allowed to use a wireless keyboard / mouse or do they have to be connected to the computer via a cable (USB)?


      Thank you in advance for your answer."


      Response of today:

      "Dear XXX,

      Please be informed that wireless devices are not allowed."

      I have followed up by asking of wireless devices indeed include wireless keyboard and mouse with reference to the above mentioned "requirements for eeqe" document. No answerso far....

      in the first an third webinar, this was not mentioned.

      Delete
    7. I remember a discussion about wired devices being more stable and reliable. And that Bluetooth was not allowed for connecting peripherals, but I assumed they were talking about speakers and microphones. Most wireless keyboards / mice are not Bluetooth - it is just wireless USB with a dedicated dongle.

      Delete
    8. And "no wireless devices" is not correct - a WiFi internet connection is allowed and a lot of printers are connected to "WiFi". I hope they clarify - this will be a disaster if true.

      Delete
    9. Wireless keyboard and mouse are allowed as an exception to the wireless devices ban. I was just contacted by a reliable source 👍.

      Delete
    10. Hi Pete,
      Where has it been mentioned/discussed that Bluetooth peripherals would not be allowed? I ask because I've been using a wireless keyboard and mouse so far as I haven't seen any reason to assume they wouldn't be allowed.

      Delete
    11. That was in the first webinar on 14 Jan according to my notes. But I understood that to be only in relation to devices for human communication (voice, sound).
      So I assume you cannot use a Bluetooth speaker and microphone, they should be wired.

      Delete
    12. I can now confirm that EPO HelpDesk has clarified the initial mail to me (see above):

      "My apologies, there has been a misunderstanding.
      Wireless devices are allowed."

      Delete
    13. I also received the following from the helpdesk:

      Dear Sir,

      The wireless mouse and keyboard are allowed.

      Best Regards,
      European Patent Office
      European Qualifying Examination (EQE)
      Support Desk

      Delete
  24. Mock3 is your last chance to test - this will be closest simulation of the real exam - if everything works, you can relax about the IT part 😊.
    23 Feb 0930-1300 CET ME (B only), 1530-1640 CET - PE (one flow).
    Even if you had no problems before, the system may have changed slightly. You may you have new hardware/software as well. No need to do the exams - at least login on time, do the ID checks, start the Browser on time, try typing for 5-10 mins, move your head around like you normally do.
    Try sending a message on Zendesk that you are taking an unscheduled break.

    ReplyDelete
  25. Hi Pete,

    I have a question.
    When a candidate is kicked off because of wrong button pressed by his finger, or other reason.
    How to re-inter in he lock down browser?
    Do the invigilator password in the real exam will still be xyz or they will give us that before the exam ?

    Do the process to re-inter the lockdown browser is the same when we loose internet connection for a long period of time ?

    Many thanks and kind regards,
    Sandra

    ReplyDelete
    Replies
    1. Hi Sandra, in the last webinar they had been very specific about that.

      When you get kicked off the browser and cannot enter without the inviligator pw you hve to contact an inviligator via zendesk...same procedure es when internetconnection is lost for long time...

      Delete
    2. Hi Anonymous,

      Thank you. I couldn't attend the last webinar unfortunately.
      There is any document explaining that specific procedure in details step by step ?
      That will be helpful!

      Do you if EPO academy will release the last webinar session so that we could listen it ?

      Kind regards,
      Sandra

      Delete
    3. EPO/epi webinars: they did not plan to make them available because they thought they were just covering the same issues again.
      I did make a screen copy of the sheets for the 11 Feb one, but unfortunately the audio recording did not work (which was the most important part). The sheets are posted in the Telegram group on 11 Feb.

      Delete
    4. Why would they think that? The webinars were oversubscribed so many of us cannot attend. If it were the same issues, it means that many candidates are worried about the same things.

      Why have a mock 3 if they have no concerns. I think it is important the EPO/EPI committee do not assume and should provide all information available to candidates. Who knows what information we are missing if things are not being openly shared by them.

      Delete
    5. I fully agree - they complained that a lot of questions being asked were already answered. Having these things online gives everyone something to go through. 1000 places for 3500 candidates is too little. A lot of people have complained, but if they did not record it, there is not much we can do.

      I think 2 reasons for Mock3 - only 50% of candidates have actually logged in and started a flow, and Mock2 highlighted a lot of problems (they did not expect it - that is why there were few invigilators)

      Delete
    6. Thanks Pete. I think it is dangerous that the EPO/EPI committee just assumes.

      We, as candidates, have been told never to assume anything especially this year. It is such a difficult year for preparing the EQEs. I understand if there is a cap on numbers attending the webinars but we were specifically told that the recordings will be made available afterwards so that others can also listen to it. I will write an official complaint to the exam sect as I don't think it is right many candidates have missed out and they decide what information to give to some candidates but not others. We are relying heavily on the goodwill from bloggers such as you Pete to digest the stream of information from the EPO.

      Delete
    7. It has also been difficult for them to get this all done in time - behind the scenes, the EQE is highly complex because of all the different parties involved.
      But the communications could have been a lot better. More centralised, and more modern.
      I thought the webinars were the best part - they answered questions, and even some live questions in the chat. I think a lot prefer this over reading because the information is presented directly and explained.
      And a technical helpdesk would have been great - it is not always easy to get the LockDown Browser to run properly. For example, one person had no problems at all, but he could not download/print documents from WISEflow. He is still waiting for a response.

      Delete
    8. Most companies in NL use Twitter nowadays for their helpdesk. It works really well - you get a response within minutes to hours and you can ask follow up questions.

      Delete
  26. Hi Sandra,

    the steps are quite similar to these Pete summarized in one of his posts here.

    Briefly: when you kicked out of the browser or your browser freezed you will have to enter zendesk in a normal browser like chrome, firefox,... and contact an inviligator explaining your situation. The inviligator may give you the inviligator pw to reenter the flow. However, in the webinar they stressed that - in general - the inviligator pw is not given to a candidate and will only be given in exceptional situation...

    ReplyDelete
    Replies
    1. Being locked out and unable to proceed is exceptional. They will give it to you - you will just lose some time.

      Delete
    2. Would it be a good idea to aim to finish each flow 10 minutes or so before the finish time, to ensure that if a late crash/ejection from FLOWLock occurs, we have time to get the password from Zendesk so we can get back in to hand the answer in before the time runs out?

      Delete
    3. My unofficial procedure is here.
      One thing that is very robust is recovery after loss of internet connection - the LockDown Browser handles this very well.
      It is better to carry on working as in many cases, internet outage is temporary (emphasised in last webinar).
      It was possible to select a different network on my system, but the Browser may decide that you are doing something that is not allowed.

      I got a response from the helpdesk: "Submission/hand in can also be done after the exam has finished. They will be able to hand in the exam but not adapt it anymore. They can do it in the space of half an hour after the exam time is finished. In case they are facing any further issues they need to contact the invigilators."
      I still need to check the official documents if this was hinted somewhere else.

      Delete
    4. Hi Pete,

      If someone is kicked off from the lockdown browser.
      1-From Zendesk, he asks for the invigilator pw.
      2-He login the lockdown browser.

      DO I will have access to my answer that was there before I had been kicked out ?

      Many thanks and kind regards,
      Sandra

      Delete
    5. Hi Sandra, yes. Once you restart the LockDown Browser, your answer will be available. As you type, the online system regularly makes backups automatically. But you need to be in the Browser to see it.
      That part works pretty well, so dont worry :-)

      Delete
  27. Hi everyone,
    I know there have been wide discussions about "having something in the ears" during the exam. However, is it even forbidden to use foamed/wax ear plugs if the neighbour decides to start drilling at the time?
    Thanks, Felix

    ReplyDelete
    Replies
    1. Hi Felix, no, it is not allowed. They could be receivers and you need to be able to hear the invigilator. Also, if your neighbor makes a lot of noise, it will be picked up by the AI.
      Can you talk to your neighbor? Or give them a copy of your exam schedule and a bottle of schnapps ;-)

      Delete
  28. Hi everyone,

    do you know if the invigilators may also ask you to scan your room during the examination or only before the start?

    ReplyDelete
    Replies
    1. In one of the webinars they said they reserve the right to ask you to do this at any time throughout the Exam. They said it can be expected in particular when you return from a unscheduled break.

      Delete
    2. So do we get an extra 5-10 minutes for them interrupting us randomly throughout the exam, especially when you are in deep thought about something - it is very distracting to be disturbed.

      Delete
    3. It will only 2 minutes. I think they will only do it if there is something strange in their monitoring, like an unscheduled break or they pick up another person on the microphone.
      I have not heard about extra time - I think not.

      Delete
  29. Do those of us with long hair have to tie it back for the exams so that our ears can be seen?

    ReplyDelete
    Replies
    1. They said in the webinar that it was not a problem.

      Delete
    2. Thank you. I was unable to attend the webinar so was unaware of this.

      Delete
  30. I wear a hearing-aid and enquired about this at the EPO - believe they discussed this at the last Webinar which I couldn't attend. Following feedback from the EPO, I had to provide them with medical evidence and also provide them with a link to the hearing aid I wear - they also informed my that information is on the Requirements documentation while making reference to earplugs for some reason when mentioned the Requirements info. Such information regarding medical evidence and having to provide a link to the type of hearing aid device is not in any of the documentation on the EPO website from what I have seen. This should be really highlighted more if it will be an issue.

    They also mentioned in the same e-mail regarding the information required that they will then inform the invigilators.

    Just an FYI for any hard of hearing candidates that wear a hearing aid.

    ReplyDelete
  31. I'm finding it super difficult to read and absorb the information on paper DII. Any tips on this. Even when I try to write things as I go along, I keep missing important things like priority is claimed from B to A - its really hard to read online.

    ReplyDelete
  32. Solution to absent chat when logged in to epo.zendesk.com:
    From Irina Bagyan: her IT administrator at the office found out that company's firewall was blocking zendesk chat, so he added following three URLs to the White list (he thinks the last one probably did it:
    ^https?://([A-Za-z0-9.-]*\.)?zendesk\.com/
    ^https?://([A-Za-z0-9.-]*\.)?zdassets\.com/
    ^https?://([A-Za-z0-9.-]*\.)?zopim\.com/

    ReplyDelete
  33. Two new docs on the eEQE page - they are now updating the REE:
    Code of Conduct
    REE Instructions to Candidates

    At the end, it says:
    43. By sitting the examination, candidates confirm that they will comply with these instructions and refrain from any behaviour that could be construed as fraudulent or that disrupts the conduct of the examination.
    So you have already agreed ��

    ReplyDelete
  34. See here my post on the Instructions. The investigation procedure seems pretty open ended.

    ReplyDelete
  35. I see that Mock 3 is accessible via WISEflow, even though emails not sent out with links as before (at least yet).

    Do you know what paper Mock 3 is for the main exam mock 3 (A, B, C or D)?

    ReplyDelete
    Replies
    1. Yes. Once WISEflow has been activated, you do not actually need the links in the emails - you can just go to WISEflow and log in.
      It will be a B paper - either from Mock1 or Mock2.

      Delete