Wednesday, February 24, 2021

e-EQE 2021 Mock3 - Experience & Tips

https://wiseflow.zendesk.com/hc/en-gb/articles/360018952959-Facial-Recognition-automated-proctoring-fact-sheet-
© UNIwise
On 23 Feb Mock3 was held as an extra test mainly for your setup: 
  • 0930-1300 CET ME (B only - not a new paper, but Mock2)
  • 1530-1640 CET - PE (one flow - not a new paper, but Mock1, Mock2)
Mock3 was your last chance to test under close-to-exam conditions. Different to Mock2, was the availability of an increased number of invigilators to help with problems and some monitoring of candidates (this picture is old - it is missing the audio/voice monitoring). See below for my comments on things that seemed new, and notes from the many messages I got before, during and after.

1. Resources (check regularly for updates):

2. Telegram group(s)
  • During and after Mock3, members of the main group exchanged tips, discussed problems in close-to-real time and helped each other. This is a very good way to beta-test software.
  • The main group has 210 members, so the conversations were getting a little chaotic. Ignacio has now added 3 more: a summer study group for EQE 2022 preparation, an (ABC+PE claims analysis) group and a (D+PE legal) group. The main group will be more for general issues, and WISEflow + LockDown Browser + Zendesk issues.
    • Mainly for students to discuss with others, but also open to any tutors. Group rules: no discussions of ways to cheat, no exchanges of large sections of copyrighted materials (excerpts, annotated examples, WIPO/EPO docs etc. are allowed)

3. Hardware / software
  • Some problems may be due to using a company computer where you don't have full admin rights. For example, blocked urls, a missing chat window in Zendesk, changing IP addresses, automatic virus scans, screensavers that cannot be disabled, updates that cannot be disabled.  
    • If you cannot solve the problems, you may have to switch to a personal laptop or PC, and either log on to the company guest network, and/or do the exam from a different location.
  • During Mock3, many candidates had problems with the camera turning off after some time during the exam (entry image and initial testing by LockDown Browser were all done successfully). It is not clear why this happens. The LockDown Browser does not generate an error or warning. It should be detected by the invigilators dashboard - if they have time and it lasts too long, they should contact you to find out.
  • If you notice the camera is off (e.g. LED turns off), report it as soon as possible in Zendesk chat (you likely will not know how long it has been off). 
  • Solution (ME): Refresh Browser (Click on browser "refresh" icon top-left). If this does not work, you may need to exit the LockDown Browser.
  • Solution (PE): For Pre-Exam, LockDown Browser works differently - you will need to enter flow AND invigilator password after a refresh before you can continue. So, you need to wait for instructions from Zendesk if you do not have the invigilator password to refresh.  If this does not work, you may need to exit the LockDown Browser.
4. Invigilation
  • Some confusion about the AI monitoring. Surprisingly, the AI does not automatically generate warnings if you move your face out of view or speak. Your camera and microphone are sampled, and the invigilators have a dashboard (see picture above) with the real time status. For example, the degree of facial recognition is monitored. If one of them is "zero" or "low" (e.g. if you move too much), you should go to the top of the list, and they may contact you. But it depends on how busy they are. 
  • When you hear the "bip" sound, you click on the Zendesk chat tab - there are no pop up messages.
  • Not being contacted does not mean that everything you did was acceptable.
  • After the exam, this data is combined with other inputs to generate a profile. All data collected is used afterwards to determine whether suspicious behaviour is suspected. The AI will generate a list of possible suspicious behaviour, which will be further analysed by people. It is the people who make the decisions.
  • Some confusion about Zendesk. You must start Zendesk chat immediately (via External Resources) after starting Lockdown Browser. It must remain activated for the whole exam.
  • However, if Lockdown Browser crashes, you can open a normal browser (Chrome etc) window, go to epo.zendesk.com and login instead. Once you restart Lockdown Browser, you need to activate the Zendesk chat again.
  • More invigilators were available compared to Mock2, and some people chatted with them. They also attempted to solve hardware/software problems with others. People asking for the invigilator password had to typically wait 20 - 30 mins. 
    • during the exam, there will be more invigilators available
    • after 30 mins, any active chat automatically timed-out
  • You can email a transcript of your Zendesk chat (before end of exam and before time-out) by clicking on three dots on bottom right and entering your email address. It gives you a full timestamped history. 
    • But if you or the invigilator leave the chat, then the next contact will create a new transcript which has to be requested separately
CLICK BELOW FOR MORE
5. WISEflow, before and after the exam
  • Some confusion about whether to show ID for the entry image. Yes, before each exam and exam part - see Step-by-Step Guide dated 12-02-21 on the official eEQE page:
    • "20. Take your entry image => You must show your ID/passport next to your face at the beginning of each flow. Once approved the photo cannot be re-taken"
  • Handed-in answer became available in WISEflow (not sure how long it took)
  •  
6. LockDown Browser
  • I could resize the assignment preview on the main tab, so that I could go through the different pages. (I remember I could not do it before, but I am not completely sure fi this has changed)
  • No change in copy/paste from assignment tab - formatting was still poor. Use CTRL-SHIFT-V to paste without formatting, or use Remove Formatting button.
    • in webinar, they promised that for the exam, it would be better
  • Assignment was still searchable using CTRL-F.
  • CTRL-TAB and similar still kick you out after 2 presses as in Mock2 (1 warning only is given)
  • Refresh of browser (using refresh icon at top left) was proposed before as a solution to problems, such as having the wrong language visible in the assignment preview. This is improved for Main Exam - no need to enter password or redo entry image. 
    • But be careful in Pre-Exam - you will need the flow password and invigilator password to continue. There it seems like the refresh function has NOT been updated.

7. Table of Contents - clickable links now work!
  • Very glad this is working.
  • Use one or two heading styles in your answer (Heading3 or Heading4 are the smallest in font size). Before applying the style, isolate the text to its own paragraph by hitting enter before the text and after (otherwise style is applied to the whole of the current paragraph).
  • In the assignment thumbnail, select Table of Contents. This will give you info about the your answer. The headings automatically generate a clickable Table of Contents which quickly jumps you around in your answer.

7. External References (EPO Legal Texts)
  • Can be very useful. But only use for specific searches - e.g. J 9/12 in Guidelines, otherwise you will waste time reading. 
  • Opens in a separate tab. Can browse or search. Use Advanced Website Search (scroll down for link) to search in specific documents and/or specific languages.
  • Clicking on hits opens a new tab. After opening some results, such as OJEPO's, there is an optional tick box at the top to "Highlight Search Terms" Large pdf's cannot be opened (like Guidelines). Smaller pdf's like forms and OJEPO notices can be opened.
  • Links not found at www.epo.org cannot be opened. In particular, WIPO links cannot be accessed. But the Euro-PCT Guide (EPG) is accessible. 
  • Also OJEPO's older than 1 Jan 2007 cannot be accessed - they are at archive.epo.org.

8. Using tables in your answer
  • Two options for making overviews - on paper, or electronically in your answer. On paper, you can run out of room. Electronically it will increase length of your answer, making scrolling difficult.
  • I tried out the table function - adding and deleting rows and columns is easy as there are shortcuts underneath. So building a table goes quite quickly. But be careful of spending too much time on the formatting. The column widths adapt automatically as you add content.
  • Realise that they will not award marks just for tables, because interpretation is often ambiguous. It is no problem to leave in your answer, but you need to include statements explaining what the table exactly means. 
  • Limitation (1): cannot select more than one cell. So you can only copy 1 cell at a time into the rest of your answer
  • Limitation (2): if you want to copy / cut / paste rows & columns, you must use the commands available in the menu when clicking on the Table icon.
  • My conclusion: if used with several columns, it can be a lot quicker than doing it on paper, and it is readable because it is typed. If making a timeline in it, you can copy rows and/or move them to make room for new values. I think I will still make A3 overviews for C and D2 on paper so that I have them visible for the whole exam. But, it could be useful for smaller overviews, such as an overview of competing rights in D2 (but make sure you explain exactly what it means).
    • if only 2 columns, it is probably quicker just to work with plain text and use the TAB key to align.

45 comments:

  1. You can copy the entire table using the source code function. It is relatively easy if one is familiar with LaTEX

    ReplyDelete
  2. "All data collected is used afterwards to determine whether suspicious behaviour is suspected."

    I guess (or actually hope) that all this data is time stamped, so it can be compared to the video recording depicting the behaviour of the candidate (during suspicious behaviour and in addition before that).

    ReplyDelete
  3. Does anyone know if it may cause any problems to use a mechanical keyboard (sounds)?

    ReplyDelete
    Replies
    1. All decisions on whether you're actually disqualified or not are made by a human in the end. I highly doubt they'd disqualify you for typing in an online exam!

      Delete
    2. I cannot see how this would be a problem. They are trying to detect voices.

      Delete
  4. Copy/paste: Does anybody know how formatting can be preserved with a Mac? The usual key conbination [alt]+[shift]+[cmd]+[v] does not work in Wiseflow Lockdown Browser.
    Even if I copy formatted text within the text editor of Lockdown Browser, the formatting is lost (I wanted to copy the amended claims to the passage where I am writing the inventive step arguments in order to see them without having to scroll).

    ReplyDelete
    Replies
    1. Unfortunately, the only way is to select the text in your answer, and click on the "I"-like icon in the menu. This strips the formatting.
      I have not heard from anyone who has done it with key shortcuts on a Mac.

      Delete
  5. Due to the inability to highlight the assignment page, I have been copying and pasting the entire exam paper into the editor and using the functionality there, as well as adding in-line notes in red.

    Do you know if there is any issue with having the full text of the exam in my answers?

    I do not expect the examiners to mark my notes or highlighting, but I wonder if there could be any negative impact on the marking of my answers (located either before or after my pasting of the exam text).

    ReplyDelete
    Replies
    1. No, there is no issue. They will just ignore it, particularly if it as the beginning or the end.
      Just make it clear where your answer starts or the exam copy starts.
      I think the main danger is you having to scroll through a lot, and you spend a lot of time formatting.

      Delete
  6. The EQE Team just sent out a link to a survey :"The purpose of the following survey is to give us your feedback on your experience of the MOCK 3 Exam and help us understand where we can improve our support to the candidates during the actual Exam in March. Thank you for your help and support. The EQE Team"
    Fill it out. Tell them about what you like , what works for you, but also where you encountered issues. I think it's important to be honest and complete. The more people respond the more the results will be representative!

    ReplyDelete
    Replies
    1. Agree. A good idea - posts on blogs can give a distorted view :-)

      Delete
    2. I think that the fact this survey can be filled in anonymously may help some to be more honest about their issues/frustrations.
      In any case this exercise seems very late in the day. 4 days before the first exam.... Are they trying to check which frequent issues should be given more leniency by the invigilators? I cannot believe that any changes to the interface or amount of printable documents is still possible or seriously envisaged.

      Delete
    3. I shedded Alt-Tab problems. It is a serious issue. :)

      Delete
  7. Thanks Pete, for you ongoing help!

    You mention when you hear a "blip" sound, to look at the zenchat.
    Is sound required? I only have a headset and my monitor has no speakers. I guess the headset is not allowed in the exam, so I have no sound hardware...

    ReplyDelete
    Replies
    1. Yes, you definitely need a speaker. It does not have to be anything special. Probably you can connect one intended to be used with a phone.

      Delete
    2. I agree with Pete. Loudspeakers required to hear voice commands / contacting of Invigilator when he/she wants suddenly to check your room, for example.

      Delete
  8. Previously in the paper version of the exam only black or blue coloured pens were allowed for writing. If I have certain text in red in my answer would that be a problem? Also, would it be acceptable to highlight with Yellow marker certain headings in my answer (for my easy reference)?

    ReplyDelete
    Replies
    1. The color was due to the copying/scanning before.
      Don't worry - use whatever highlighting/formatting you need (you will see it retained in the copy of your own answer you get afterwards).
      They can easily remove it digitally if needed.

      Delete
  9. In this e-EQE format, we only have one editor which we use for highlighting in the "exam pdf" but also for writing down our answers. Do we need to delete everything that's not an answer before we hand in? I've yet to get a satisfactory answer from any of the e-EQE documentation.

    ReplyDelete
    Replies
    1. I have seen nothing about any restrictions. If you clearly mark it as Exam (at the end), or you have a heading Answer (if you have the exam at the beginning), they will ignore it. It gets no marks at all.
      If you have time, you could also delete it, but you probably want to see it in the copy you get it.
      Don't worry about it. They can also delete it before sending it for marking.

      Delete
  10. Touchpad: FAQ of Febraury 21 state that use of touchpad gestures is not allowed - seems to be a reaction of the LockdownBrowser crashing in response to some (unknown) gestures.
    In order to avoid any technical problems with camera, admin rights etc. I decided to write the exam with my Macbook with nothing else connected, although the screen is much too small. But technically everything worked fine for mock2 and mock3 and reference photo. However, the Macbook has a touchpad and I use 2 fingers for scrolling. I did not get any warning during mock2 and mock3 but I am now wondering if the scrolling via the touchpad might be (since not allowed according to the FAQ) a problem afterwards.
    Honestly, I do feel completely helpless meanwhile.

    ReplyDelete
    Replies
    1. The EPO is very careful in stating what is allowed, and if there is doubt, they state that it is not allowed. There are so many different systems and configurations to take into account.
      The problem with touchpads are that they can have many hidden features that you never use, but can accidentally trigger (I discovered during preparation for the mocks that I can zoom in and out by 2-finger pinching, and by default 3 and 4 finger gestures were turned on)
      In principle, a scrolling-only gesture is allowed by the software, but not any gesture that switches windows or applications. However, there is no clear list exactly what the LockDown Browser triggers on.
      So either:
      - go into settings and disable all gestures except scrolling. Likely you will be okay, but this is still a risk.
      - use an external mouse with a scroll wheel. This is the lowest risk. Wireless is also allowed.

      Delete
  11. Does anyone know if we can use our own indications for the papers. There is no suggestion that they should ignore any keys that you've submitted. For example, if you don't use the [] to indicate deletion in Paper B but you put a key to say ** as your way of indicating deletion, will they ignore this. I only put this because I got used to it during MOCK 1 before they announced how to provide deletion and then practiced using this. It crept it during mock 2 (accidentally) but now I worry if it does creep in the main exam, will I get penalised for this. Noting that they didn't give us much time before announcing changes to Paper B.

    ReplyDelete
    Replies
    1. You will not lose marks for something different, as long as it is clear. You will also explain in your argumentation something about it.
      They will understand - don't worry!

      Delete
  12. Hello. I contacted the Secretariat, to find answers to important questions:
    - via "external ressources" what documents will be accessible, in particular the advanced search will it be accessible ? In particular PCT documents ? The FAQ answer is not clear
    - will D2 become printable before the start of the flow ?
    These are truly essential information, way more than knowing details about the face recognition algorithm !!
    Best regards, François V
    -

    ReplyDelete
    Replies
    1. Seek, and you shall find:
      - Do I have access to the legal texts in electronic form?

      In the LockDown Browser, you will have access to the EPO legal texts as "external resources" (see user guide point 6.4)

      You will be able to navigate the EPO legal texts webpage. No other legal texts will be made available online.

      When consulting the legal texts, do not download the large pdf documents. Please navigate to the html version and the smaller pdfs only.

      (https://www.epo.org/learning/eqe/faq.html)

      During the webinars, they stated that only links which start with epo.org are accessible. So the WIPO website is not, and large PDFs are stored in a different domain than epo.org.

      - "Candidates will be allowed to print only the calendar. For that purpose, the calendar will be made available well in advance before the start of paper D."
      http://documents.epo.org/projects/babylon/eponot.nsf/0/66EB601464EC7BECC12586320050164F/$FILE/ExamPapers%20EQE%202021_EN.pdf

      Delete
    2. I suppose that the calendar in the link and cited by the EPO 2021 is the official calendar to be used also in the Exam. Actually, I tried to confirm this in a couple of years ago but I didn't get any confirmation from Helpdesk!! Then, there was some warnings in Deltapatents webpage that to use only the calendar provided in the Examination! This definitely, at least then, was very indefinite! :)

      Copying of the whole guidelines-PDF takes place successfully but when pasteing that to LockDown it will freeze the browser!

      Delete
    3. I have noticed that the advanced search is not so powerful when compared at least to the Google. With the same keywords Google finds the relevant chapter from the Guidelines when by the same keywords, the advanced search doesn't find anything! Zero hits.

      Delete
    4. Thanks Pete. Just such as I remembered.

      "For future examinations, candidates shall answer the papers solely on the basis of the calendars in the examination papers."

      So no useful to preprint.

      Delete
    5. No, sorry, I did not mean that. It also says that the calendars will be made available in advance, so you can already print the ones from the EPO EQE notices page. They are the calendars that they used.

      Delete
    6. Ok! Thanks again.

      Delete
  13. This comment has been removed by the author.

    ReplyDelete
  14. I suppose that not being able to print D2 means that the process is likely to be:
    - copy/paste all D2 in the editor
    - rephrase it i.e. draw conclusions (priority claim valid or not), entirely to make sure all information is used
    - study situation us/them
    - study options (div, add priority claim) etc.

    Do any reader of this logic agree with me ? Advices ?

    In itself the e EQE is questioning the exam itself: core competencies in ctrl F, ability to reverse engineer the examens, de-puzzle C, etc etc. The Case law barely knows about the existence of search engines and inventive step incentives. Not talking about transposition and analogy thinking ...This once more shows the exam is obsolete.

    Vive qatent.com !!!

    ReplyDelete
  15. Hi,
    I’m preparing to sit the pre-exam on Monday. Does anyone know whether, in calculating time limits, we should take into consideration the interrupted period 15 March-2 June 2020 due to Covid disruption?
    Thanks!

    ReplyDelete
    Replies
    1. They will need to tell you the dates of interruption in the question.

      Delete
  16. Pete, Complete PCT Applicant's Guide just arrived yay! I was thinking, however, it would list the languages accepted by the ROs, am I wrong? I don't feel like printing that bloody 500 page thing, thinking of making an educated guess if it comes up ;)

    ReplyDelete
    Replies
    1. Hi Leigh, thanks for buying the book. It arrived just in time :-).
      Sorry, that overview is too difficult to keep up to date. If you look in the Annex C's, you will see the languages. As you can guess, they accept their official languages (in general). But there are also many that also accept English, like CN. Also all EPC states allow at least one official EPO language.
      It is so stupid that they cannot arrange the access from the exam.

      Delete
    2. Hi, Pete, yes, I am excited! Thanks! I'll either print the relevant pages on Monday or just take educated guesses "This RO accepts filings in DE and EN. If they don't, it will go to the IB" ;)

      Delete
    3. And the other useful overviews are in PCT References Part 1: Overviews.

      Delete
  17. Does anyone know if we are allowed to use heaters during the exam? My room gets freezing cold these days and I only have a fan heater but it makes a lot of noise. It would be really horrid to have to do it without the heater but I don't want to risk getting DSQ...

    ReplyDelete
    Replies
    1. If it is way out of reach, on the other side of the room, it should be fine. Heating is part of a normal room.
      If you have it too close, or blowing directly towards you, it will be a problem because of the ambient noise swamping the microphone. So direct it up or away.

      Delete
  18. Hi Pete. Thanks for this. During Mock 2, at the end of the allotted time, a pop up window came up, indicating the time expiration and asking whether I wanted to hand in. Is this going to be the case in the real exam too? Will I be asked whether I want to hand in, or should I be careful to hand the paper in before the end of the allotted time? Thanks.

    ReplyDelete
    Replies
    1. As far as I know, this should be the same. I would definitely try and hand-in 1-2 minutes before the end in case you have technical difficulties.

      Delete
  19. On the official e-EQE page:
    Message from the Examination Board (3 Mar 2021)
    The Examination Board of the EQE is aware of a situation that affected paper D1.1. The Examination Board guarantees that the marking process will be conducted so that no candidate will be disadvantaged because of that.

    ReplyDelete