GSoC 2018 Final Evaluation - VLC macOS interface redesign

2018-08-09

Hi all,

This blog post contains all relevant work I did in summer with VideoLAN on “macOS interface redesign” project.

Title of the project

VLC macOS Interface redesign

Synopsis

The project aims to revamp VLC’s user interface for macOS and improve the user experience. It introduces the media center (aka. the libraryView), which provides users with a whole new way to browse and play video files on their computers. A new control system for video playback was also developed.

My task as a GSoC Student was to implement a libraryView or mediaCenter view inside VLC with Service Discovery (loading videos into the mediaCenter) with help of CollectionView.

The Team

From left: Vibhoothi, Daksh ,Jean-Baptiste ,Felix , David

The VideoLAN community as a whole was really helpful responding very positively to my queries on Freenode #vlc,#vlc-ports,#vlc-gsoc.

What I did

After been accepted at VideoLAN as GSoC 208 student I first worked on an Icecast Patch,“Add Icecast streaming to streaming gui” that helped me learn more about the code base and the standards and more importantly coding conventions that are used which I was not familiar.

During first evaluation part, I had my University Exam so I wasn’t active first  1 and half weeks.After my exams on 29th of May, I have resumed work, as most of the time of day I was working,I think I worked more than 10 hours on most of the days as I was mostly idle and  had no other work to do, even though I took few hour breaks in between to watch some TV series,

The patch was about introducing an Icecast streaming option into the Streaming GUI of Convert/Save, Following things was done as part of the patch which can be seen in “icecast#8” branch of vlc/gsoc/macosx,

• Introduce Icecast Streaming Service #1 Introduced by adding new fields with proper auto layout constraints like mountPoint, username, secure password field, Initialised Icecast functioning part inside streamTypeToggle function with proper debug messages. I implemented these new fields with Auto Layout constraints so that they can resize dynamically

• Introduce Icecast Streaming Service #2 In modules/access_output/shout.c, I added a NULL check for the mount point to p_shout, and debug logs.

• Introduce Icecast Streaming Service #3 Earlier Icecast patch had some issues when using arabic translations and Spanish where the text length overlapped the input fields which didn’t go well so I had to redo most of the constraints.

During the first week of June, I flew to Frankfurt, Germany for a face-to-face meeting with the team to finalise the plans we wanted to do. After the meeting, I have started working on implementing the libraryView starting off first by removing the sidebar,splitView and podcasts from VLCMainWindow cocoa class. For refactoring the codebase I had to first understand the full working of the _VLCMainWindow _class, Fortunately, in the face-to-face meeting, the team clarified my uncertainty about what I needed to remove and keep.

The following things were also done as part of first steps for making collectionView

• Remove podcast from VLCMainWindw.m Removed the podcast section, which will no longer exist in version 4.0, from the sidebar. Things like Podcast Views, Podcast Subscribe Window, Podcast unsubscribe window, Podcast controls, Podcast Buttons (addPodcast, removePodcast, addPodcastWindowAction) and functions to remove show/hide podcasts (showPodcastControls, hidePodcastControls).

• Remove sidebar from VLCMainWindow. The sidebar is removed as a whole in favour of collectionView which will be replaced and which will be the default view when player loads. For this,we removed the code related to the sidebar and functions_restoreSplitView,_ _reloadSidebar _functions and also sidebarHandling, which will be deprecated in VLC 4.0.

• Remove splitView from VLCMainWindow.m Methods and variables related to splitView  like NSSplitViewDelagate,  _makeSplitViewVisible, makeSplitViewHidden etc. The most important and difficult part was scraping residues of splitView from changePlaylistState which was a bit complex.

• Remove splitView from Interfacebuilder MainWindow.xib Here we have removed all labels, icons, columns, constraints and properties related to splitView and removed dropZone view Only and kept the codes.

After refactoring the code Felix helped me to refactor the more complex parts. Also he added an empty basic collectionView (adding a basic NSCollectionView) to adapt further, which helped me to make smoother progress.

Then I started playing with the collectionView which was the end of the First Phase of the project.

The second phase of the project was more like making a proper collectionView or libraryView advance using modern APIs

basic libraryView using Hardcoded images You can see below, when we open VLC it has the collection view instead of the regular sidebar and splitView. The first implementation was like having an NSArray with collectionView. The view for the collectionView was provided by NSCollectionViewItem cococa class with interface builder. After adding that inside the collectionView, the image is passed to the array using representObject and valueForKey

[self.itemImageView setImage:[[NSBundle mainBundle] imageForResource:[representedObject valueForKey:@"itemImage"]]];

The current VLC 3.0 View

Very First Prototype of LibraryView using  hardcoded images

This method had so many issues. It was not mutable and hardcoding images were not possible for the real implementation. It was also not using modern APIs of collectionView which was introduced since macOS 10.11.

• Advance CollectionView using modern 10.11 NSCollectionView APIs

So as part of making an advance collectionView, we raised the minimum OS requirement of VLC from macOS 10.10 to 10.11. This allowed for very flexible collection view with advanced properties like NSFlowLayout, collection view delegates. It also has very useful APIs like “didSelectItemsAtIndexPaths”, “numberOfItemsInSection”, “itemForRepresentedObjectAtIndexPath”, and “numberOfSectionsInCollectionView”, which will make the implementation of the libraryView simpler and cleaner.

The main problems trying to delegate the following NSCollectionViewDelegate protocol as an object. Earlier I used mouseDown to track any selection states, but I later figured out that the cell should not handle this state but the NSCollectionView and its delegate instead. Earlier I had been using drawRect to colourize the NSView, but later did it AppKit where I just had to setup the background colour and rest will be done nicely by AppKit itself. Also, we made several changes to VLC Xcode Project to adapt the libraryView

Few notable things in the advancement part are below,

• extras: Introduce new properties for VLC Xcode project

This commit introduces various changes to the current VLC Xcode project which was essential for future development. Also, few are done as per Xcode Specification

• macosx:Introduce VLCMainWindowCollectionViewItem cocoa class with XIB

This introduction is for making libraryView based on NSCollectionView by default for VLC 4.0

The following files are newly introduced

• VLCMainWindowCollectionViewItem.xib : An Interface for showing collectionView items.

• VLCMainWindowCollectionViewItem.h : Header file for defining instance and instance variables

• VLC MainWindowCollectionViewItem.m: Cococa class which contains the implementation of instance and representation of objects

Build VLCMainWindowCollectionViewItem files

• extras: Hello macOS 10.11

From VLC 4.0 we are making 10.11 macOS version as the minimum specification for having libraryView feature and for all who are below 10.11 will have normal VLC without libraryView feature.

• MainWindow: Introduce libraryView

• Loads the image from default location which is hardcoded as of now using NSColelctionVIewDataSource and NSFileManager.

• Uses FlowLayout for showing the items in the collectionView

• Shows Highlight of selection with blue colour (alternateSelectedControlColor).

• Get the index of selection using delegate following NSCollectionViewDelegate protocol and for debugging and all have displayed the log of it to command line.

• Scrolling function is also there with this, when the user resizes Window, it dynamically resizes the collection view items

Sample Prototype 2 of libraryView using Advance macOS APIs

These things spanned up through the midterms.

In the Third and Final phase was the real implementation of the libraryView.

The first problem in that I faced was the wrong view of libraryView due to Autolayout constraints, to fix up those I had to redo and recheck all constraints, as time was bit low so I kept fixing constraints to TO-DO list and continued working on implementing the popOver

libraryView W/O proper Constraints

libraryViewItem with wrong constrainsts

So for Implementing the popOver which shows metadata I used NSPopOver linked to a button inside collectionViewItem

Due to uneven or wrong constraints, the view was super ugly and not at all organised. Also, I was not able to fix the issue at first because it was confusing for me. So again I kept this on my To-Do list.

To display video inside the collectionView or libraryView, I made a fresh project where I implemented a video collection view using AVAsset and generated video thumbnails using AVAssetImageGenerator.In a separate standalone Xcode project where I have made a simple window(NSView) with NSCollectionView and NSCollectionViewItem coupled with image, string and a button. After trying a few times I was able to generate a sample with multiple NSMutableArray’s to store values of the dummyData which was generated.

Now the state of player was like I could load videos from a specified directory in which a popOver with dummyStrings in a nice way. The two things kept in TO-Do List (Fixing Constraints issue of the CollectionViewItem and making the popOver not working) was also fixed by this time

The most confusing and trouble was implementing the MVC(Model-View-Controller) to the current Implementation it took me a bit longer to make it fully working, had a series of discussions and different approaches to make it possible working so there will be a good separation of the model, view and the controller. When I tried to implement the MVC on the collectionView, I have faced multiple issues at different times. The final working approach for the dataModel was like Make a VLCLibraryItem class and define the data models,then use that the data model class to structure your data and your data source object (VLCMainWindow) will use it to provide data to the view after that change the values of this class object to subsequent calls. First I made three different arrays to store details like image, videoTitle, years, labels then work was like add dummy data to NSMutableArrays then pass those to dataModel after that call function “assignValueForDataModel:(VLCMainWindowDataModel *) dataModel” where dataModel is passed as an argument so inside this function values from the dataModel is assigned to the properties of view properly.

The issue with this approach is that it is having more than one instance of dataModel (multiple Data Structures), by the concept of MVC I should have only one dataModel. The potential solution to this problem is making one dataModel object then assign the values which were generated from the prepareData function to objects of the dataModel class. Again another issue which I faced after doing this is that only last element was added or the same object of dataModel was added repeatedly to the array. After a quick discussion with David it was reached to a conclusion that reallocating the array every time after adding to the array( self.dataModel = [[VLCMainWindowDataModel alloc] init ] ) this fixed the issue of adding dataModel to the array. Last issue regarding the MVC implementation was assigning the dataModel to relevant objects in the collectionView or libraryView, for this also I tried using loop but all gone into vain. Finally using objectAtIndexPath fixed the issue, all these tries was inide itemForRepresentation.

VLCLibraryItem *model=[self.dummyData objectAtIndex:indexPath.item];

The following are some notable and important changes as part of the final phase:

This commit introduces a minimal popOver with a button into the collectionView (aka. LibraryView), which shows all metaData information of the video when the user clicks the button.

Fully revamp the current implementation of the libraryView with proper Guidelines of MVC so now there is a dataModel which is passed to VLCMainWindowCollectionViewItem with assignValueForDataModel with dataModel as an argument.

Make and use only one instance of dataModel for showing items inside libraryView This also fixes the regression of 264754b3 . Also worth noting is that the loading performance increased marginally after this patch.

Introduce thumbnails to VLC’s libraryView which is now loaded from a local directory. This commit also introduces videoTitle field below to video thumbnail in libraryView

To wrap up the final phase work, I present the final prototype:

Final Prototype of the libraryView

This wraps up my summer work. It was really great and I had an immense learning experience. I would like to express my sincere thanks to both David and Felix for constantly supporting and guiding me when I was stuck. Even when I brought up very basic doubts they were kind to explain them to me. Also, I would love to say special thanks to Jean Kempf(j-b), ePirat and all others from VideoLAN community who helped me. Last but not least, I would like to thank everyone who sent me their good wishes

My Next Plans after GSoC

After the GSoC, I would really love to continue working on this project and maintain the libraryView and all the things which I did for the community. From the project proposal and from the face to face meeting we had some ideas like introducing a list mode which is found on iOS or formally known as the Play-Queue,Implementing a Audio Player coupled with collectionView, having a seperate section for Internet where users can play videos from YouTube,Vimeo etc and lastly and most important part which was outside scope of project, implementing the libraryView using libVLC core libraries and APIs instead of the dummyData with nice animations and blending option to make the libraryView ready to be shipped with VLC 4.0.

Currently I have only contributed to small parts of the VLC media player, I am planning to contribute to multiple platforms and multiple platforms where VLC is available from end of 2018.

Skills Acquired

• Objective C: I feel now I am comfortable doing stuff with objective C
• Learned how to work with the large codebase
• Also became familiar and understood many parts of the VLC source
• Learned time management and meeting the deadlines and also GSoC helped to stop procrastination up to an extent
• Became flexible using the Xcode environment.
• Learned to how to debug applications crashes and much more

Best Regards,

Vibhoothi